-- Logs begin at Thu 2022-08-04 01:15:35 UTC, end at Thu 2022-08-04 04:04:12 UTC. -- Aug 04 03:20:17.575502 dv-cinder-733062-32-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 03:20:17.582853 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 03:20:17 2022] *** Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: nodename: dv-cinder-733062-32-iscsi Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: machine: x86_64 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: clock source: unix Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: pcre jit disabled Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: detected number of CPU cores: 4 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: current working directory: / Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: detected binary path: /usr/bin/uwsgi-core Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: your processes number limit is 47680 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: your memory page size is 4096 bytes Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: detected max file descriptor number: 2048 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: lock engine: pthread robust mutexes Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: thunder lock: enabled Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 03:20:17.587872 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 03:20:17.625155 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: Python main interpreter initialized at 0x55f4ea2ca9c0 Aug 04 03:20:17.625155 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: python threads support enabled Aug 04 03:20:17.625155 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: your server socket listen backlog is limited to 100 connections Aug 04 03:20:17.625155 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: your mercy for graceful operations on workers is 90 seconds Aug 04 03:20:17.625530 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 03:20:17.625604 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: *** Operational MODE: preforking *** Aug 04 03:20:17.625876 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: *** uWSGI is running in multiple interpreter mode *** Aug 04 03:20:17.625876 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: spawned uWSGI master process (pid: 80701) Aug 04 03:20:17.626077 dv-cinder-733062-32-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 03:20:17.626886 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: spawned uWSGI worker 1 (pid: 80713, cores: 1) Aug 04 03:20:17.626974 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: spawned uWSGI worker 2 (pid: 80714, cores: 1) Aug 04 03:20:17.629260 dv-cinder-733062-32-iscsi devstack@keystone.service[80701]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 03:20:23.189034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 03:20:23.189034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: from cryptography.utils import int_from_bytes Aug 04 03:20:23.189712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 03:20:23.189712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: from cryptography.utils import int_from_bytes Aug 04 03:20:23.190146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 03:20:23.190146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: from cryptography.utils import int_from_bytes Aug 04 03:20:23.190275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 03:20:23.190275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: from cryptography.utils import int_from_bytes Aug 04 03:20:23.239837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 03:20:23.240014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 03:20:23.240109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 03:20:23.240212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] command line args: [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 03:20:23.240281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 03:20:23.240417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] command line args: [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 03:20:23.240506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 03:20:23.240624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ================================================================================ {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 03:20:23.240708 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 03:20:23.240847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ================================================================================ {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 03:20:23.240930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] admin_token = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241097 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] admin_token = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] config_dir = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] config_dir = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] config_source = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] debug = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] config_source = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.241914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] debug = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242854 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.242923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_config_append = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243471 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_config_append = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.243934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_dir = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_dir = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_file = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_file = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.244853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.245988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.246983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.247894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] publish_errors = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] publish_errors = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.248985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] standard_threads = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] standard_threads = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] use_journal = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] use_journal = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] use_json = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] use_json = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] use_stderr = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] use_stderr = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] use_syslog = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] use_syslog = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 03:20:23.250648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.250715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.250825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.250981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.251918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.252897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.253978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.254143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.254226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.254389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.254472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.254626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.254703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.255681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.255770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.255984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.256987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.257963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.258926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.259945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.260896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261722 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.261942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.262952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.263859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.264926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.265967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266195 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.266908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.267846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.268952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.269985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.270923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.271906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.272853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.273934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.274949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.275896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.external = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.external = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.276996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.password = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] auth.token = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.password = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] auth.token = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.277953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.278971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.279921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.280866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.281849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.282871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.283941 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.284963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.285945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.286635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.286734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.286873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.286952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.287923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.288848 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.289970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.290949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.291958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.292888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.293926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.294876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.295954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.296974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.297925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.298860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.299969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.300963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.301114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.301191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.301363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.301442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.301636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.302213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.302434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.302515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.302688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.302766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.302938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.303862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.304960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.305927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.306810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.307961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.308986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.309927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.310962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] role.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] role.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] role.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] role.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.311985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.312878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.313934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.314952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.315952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.316963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.317976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.318951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.319930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320875 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.320953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.321925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.connection = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.322887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.connection = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.323863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.324934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.325978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.326999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80714) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 03:20:23.327077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.327287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.327522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 03:20:23.327692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 03:20:23.356461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.356600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.356730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.356827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.356904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.357040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.365828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.365997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.395316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.395654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.395739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.395917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.448152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.448437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.498330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.498560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.499829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.500067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.563942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.564062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.564143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.564277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.564492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.586823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.587962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.588080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.588159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.588292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.588508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.599501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.599764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.612997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.613077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.613224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.613308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.613449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80714) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.613530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.613711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 03:20:23.630922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.631134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 03:20:23.631217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.631370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 03:20:23.631450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.631651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 03:20:23.631747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.631888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 03:20:23.631969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.632905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.642289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.642561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.642643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.642824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.642906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.643102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.643191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.643377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.650264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.650384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.654944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.655198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.655293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.655481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.655561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.655749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.655828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.656884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.657929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.681882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.682691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.683357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.683631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.684183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.684478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.688218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.688494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.688769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.689183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.689484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.689763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.695484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.695772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.696559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.696841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.699919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.700160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.700434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.700719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.700947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.701028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.701249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.701334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.701534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.701615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.701815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.701906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.702089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.702382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.702652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.702919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.718289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.718562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.719426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.719705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.722479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.724201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.726226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.726518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.727522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.727816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.730899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.732228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.734128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.735450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.737291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.737594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.737876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.738166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.738438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.738632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.738720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.738940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.739304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.739659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.739945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.740224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.754704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.755249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.755512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.755779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.756506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.756789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.757052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.757336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.762686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.764188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.764326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.764453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.765751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.766026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.767290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.767564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.767907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.768186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.768458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.768842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.769123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.769442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.769722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.770000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.780374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.780648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.780936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.781215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.781490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.781757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.782016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.782153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.782279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.782552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.782831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.783110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.783392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.783674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.798220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.799870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.801184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.802913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.804176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.805868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.806656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.808126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.808384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.808478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.808670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.809898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.810186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.810473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.815143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.816732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.816919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.818752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.819815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.820098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.820378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.820651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.821711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.821999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.822278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.822550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.828681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.828976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.829277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.830609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.830904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.831179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.836680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80714) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.836954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.837252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.837527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.837800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.838065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.838330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.838601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.838725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 03:20:23.838980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.839863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.840032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.840118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.840293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80714) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.840381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.840572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.840957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.841248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.841523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.841782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.842044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.842318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 03:20:23.874465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f4ea2ca9c0 pid: 80714 (default app) Aug 04 03:20:23.876470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f4ea2ca9c0 pid: 80713 (default app) Aug 04 03:20:23.877756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30b9c-05ef-4c74-b30c-536b5f05bd1b None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:23.878044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30b9c-05ef-4c74-b30c-536b5f05bd1b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:23.878317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a30b9c-05ef-4c74-b30c-536b5f05bd1b None None] PATH_INFO: `/v3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:23.883003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Thu Aug 4 03:20:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:32.343343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:32.343841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:32.343921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:32.348352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1/2] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:32.353808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:32.354208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:32.354381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:32.424946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: INFO dbcounter [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] Registered counter for database keystone Aug 04 03:20:32.445224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG oslo_db.sqlalchemy.engines [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80714) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 03:20:32.451564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writer thread running {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 03:20:32.631479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG passlib.handlers.bcrypt [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80714) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 03:20:32.631890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG passlib.handlers.bcrypt [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80714) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 03:20:32.686604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:32.688167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:32.817870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 2/3] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 03:20:32.958632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:33.028534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: INFO dbcounter [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] Registered counter for database keystone Aug 04 03:20:33.049581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG oslo_db.sqlalchemy.engines [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80713) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 03:20:33.056136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writer thread running {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 03:20:33.178211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104482-69db-49f5-bab7-d6de5a7f703e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:33.182486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:33.183120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:33.183398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:33.183669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:33.191687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:33.245006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 03:20:33.245316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba90527-c4d9-4d0f-8eee-040f295a8683 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:33.256242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 2/4] 10.222.0.66 () {58 vars in 1324 bytes} [Thu Aug 4 03:20:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 03:20:33.262245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:33.276613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfcf10-8756-439d-8568-3fddefc53726 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:33.281048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:33.282191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:33.282476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:33.282751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] PATH_INFO: `/v3/projects/94f13981caf746e383601ed3304a94e7` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:33.284230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] RBAC: Authorizing `identity:get_project(project_id=94f13981caf746e383601ed3304a94e7)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:33.334847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 03:20:33.335197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e06028-1515-4e25-9284-018c6c81cada admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:33.339828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 3/5] 10.222.0.66 () {56 vars in 1207 bytes} [Thu Aug 4 03:20:33 2022] GET /identity/v3/projects/94f13981caf746e383601ed3304a94e7 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:35.325920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7a43-0710-4ed3-a451-8f73b44bc1d3 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:35.326417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7a43-0710-4ed3-a451-8f73b44bc1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:35.326527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce7a43-0710-4ed3-a451-8f73b44bc1d3 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:35.330591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 3/6] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:20:35.336933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:35.337341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:35.337524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:35.357710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:35.359235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:35.367787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 4/7] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 03:20:35.390336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7a43-0710-4ed3-a451-8f73b44bc1d3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:35.402410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce7a43-0710-4ed3-a451-8f73b44bc1d3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:35.406670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360d06b-c55c-4ecf-b12d-05cd07b3f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:35.407271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360d06b-c55c-4ecf-b12d-05cd07b3f149 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:35.407583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360d06b-c55c-4ecf-b12d-05cd07b3f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:35.407858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360d06b-c55c-4ecf-b12d-05cd07b3f149 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:35.415674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360d06b-c55c-4ecf-b12d-05cd07b3f149 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:35.417531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8360d06b-c55c-4ecf-b12d-05cd07b3f149 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:35.426308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 4/8] 10.222.0.66 () {58 vars in 1324 bytes} [Thu Aug 4 03:20:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:20:35.432051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:35.441069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3320806-6f2e-46f4-9338-b0847ebd9aa1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:35.446345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20308475-5415-4c98-981e-cf7e2201a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:35.446971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20308475-5415-4c98-981e-cf7e2201a37c admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:35.447253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20308475-5415-4c98-981e-cf7e2201a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:35.447526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20308475-5415-4c98-981e-cf7e2201a37c admin admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:35.448889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20308475-5415-4c98-981e-cf7e2201a37c admin admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:35.450773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20308475-5415-4c98-981e-cf7e2201a37c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:35.452299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 5/9] 10.222.0.66 () {56 vars in 1198 bytes} [Thu Aug 4 03:20:35 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:37.500676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda4b3-6f31-4545-8748-0ed8a749b243 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.501222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda4b3-6f31-4545-8748-0ed8a749b243 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.501222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda4b3-6f31-4545-8748-0ed8a749b243 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.502513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 5/10] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:37.508278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.508716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.508844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.526834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c06b-8305-40f9-98f1-e19c60dc5214 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.527119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c06b-8305-40f9-98f1-e19c60dc5214 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.527394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3c06b-8305-40f9-98f1-e19c60dc5214 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.528713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 6/11] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:20:37.528944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:37.530189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:37.533728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.534011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.534285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.576553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG passlib.handlers.bcrypt [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80713) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 03:20:37.576977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG passlib.handlers.bcrypt [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80713) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 03:20:37.612390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 6/12] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:37.623236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:37.624832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:37.636205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:37.659871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:37.727321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 7/13] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:20:37.741488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319f5dd-5265-42a7-8a76-0a374631d98a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:37.749122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:37.776435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:37.777010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.777321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.777596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.789096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:37.797664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:37.798564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:37.799384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 7/14] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:20:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:20:37.806682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:37.851037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60083f-2f7f-4046-9710-ae99c40a073c None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:37.854110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:37.888589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:37.891121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:37.891991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.892272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.892544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.906807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704381aa-db48-4299-a3a3-9105e1c44703 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:37.940854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:37.941529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:37.941816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:37.942677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:37.943406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:37.943738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:37.945152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:37.956955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 8/15] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:37.960543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:37.962101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:37.962454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:38.059519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 8/16] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 03:20:38.061231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea53c28-79a2-452b-8cd2-cad621ca09ca None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:38.065032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:38.068837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:38.095938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:38.096578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:38.096867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:38.097146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:38.098950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:38.100533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:38.102171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 9/17] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:37 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:38.107452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:38.122425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:38.164018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e869625-47f2-410f-9446-17b7c703d320 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:38.200260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:38.201127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:38.201440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:38.201716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:38.212063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bfc6f-0bad-4213-a478-144ab6efdf09 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:38.219839 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:38.221401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:20:38.241957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:38.242928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:38.243746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:20:38.243746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:20:38.243746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:20:38.243746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:38.243746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 9/18] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:20:38.249524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:38.310791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:38.354043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:38.354692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:38.354976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:38.355249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:38.355395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b013e-7a4e-4317-942b-db051c72abf0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:38.356396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:38.357894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:38.359478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 10/19] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:38.365769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:38.390306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:38.390978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:38.391313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:38.391585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:38.392275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:38.392578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:38.393995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a476d2-daed-4433-81f2-22c097f41009 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:38.412917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 10/20] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:38.449935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:38.470704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabef87-c98a-4722-ae6a-61079e132d37 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:38.507083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:38.507723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:38.508007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:38.508285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:38.509093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.domain.id=default, user.id=4d0f702c6dd04ac1a8cf16b1ca420f19) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:38.509641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:38.510905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05babe8-a8b2-4b72-90c6-8cade9400e44 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:38.518513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 11/21] 10.222.0.66 () {56 vars in 1322 bytes} [Thu Aug 4 03:20:38 2022] GET /identity/v3/role_assignments?user.id=4d0f702c6dd04ac1a8cf16b1ca420f19&scope.domain.id=default&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:40.956188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a77f0-0721-437e-a940-b5e6115b6704 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:40.956713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a77f0-0721-437e-a940-b5e6115b6704 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:40.956713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a77f0-0721-437e-a940-b5e6115b6704 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:40.958300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 11/22] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:40.975204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c4d60-bbaf-4ab7-8d61-44d60803d800 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:40.975642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c4d60-bbaf-4ab7-8d61-44d60803d800 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:40.975812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c4d60-bbaf-4ab7-8d61-44d60803d800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:40.991454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877921c1-f7a3-40bd-ac9c-a66e3e836d9d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:40.991730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877921c1-f7a3-40bd-ac9c-a66e3e836d9d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:40.992011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877921c1-f7a3-40bd-ac9c-a66e3e836d9d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:40.993392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 12/23] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:20:40.998507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:40.998790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:40.999066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.016747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-539c4d60-bbaf-4ab7-8d61-44d60803d800 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:41.018159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-539c4d60-bbaf-4ab7-8d61-44d60803d800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:41.030702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:41.031904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:41.141795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 12/24] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:41.143158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736bfd6-7430-4308-b018-09e4e6ece7b6 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:41.143450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736bfd6-7430-4308-b018-09e4e6ece7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:41.143728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736bfd6-7430-4308-b018-09e4e6ece7b6 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.145707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 13/25] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:41.167396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:41.172214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 13/26] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:20:41.172356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:41.172923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.199181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:41.200425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:41.202026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:41.307991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:41.318416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 14/27] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:41.319689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:41.339324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:41.390998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400f218-48b4-46d7-bd07-9186781d0330 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:41.456843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:41.459129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:41.459889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:41.461329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.480664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:41.483523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:20:41.511472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:41.512287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:41.513068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:20:41.513068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:20:41.513068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:20:41.513068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:41.518207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 14/28] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:41.520995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:41.525589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0a98f-be76-4bc7-82bf-05e58e4a1623 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:41.574806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:41.575894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:41.576374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:41.576823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.589190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:41.595136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:41.595946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:41.596663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:41.600522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 15/29] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:20:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:41.602389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:41.730145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:41.781948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:41.787011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3331e-2177-4829-b0f6-a35809746f17 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:41.821034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb74a5-50c1-40e1-9ec9-c299e5beaf77 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:41.851667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:41.852568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:41.853030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:41.853505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.854427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:41.856062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:41.869826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:41.870468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:41.870749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:41.871027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:41.871731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:41.872038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:41.873473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:41.892637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 16/30] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:41.897233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.011604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26e210-b2d6-4b49-9a0b-2dab81a6a3e0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.045606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:42.057703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.058338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.058625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.058901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.059639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:42.059945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:42.061496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4a913-da0a-4d45-8131-a239cfd38919 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:42.071736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 17/31] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:42.073056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e29f7-2302-4aa7-ab28-04426045c82d None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.073384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e29f7-2302-4aa7-ab28-04426045c82d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.073671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e29f7-2302-4aa7-ab28-04426045c82d None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.075173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 18/32] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:42.075728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e29f7-2302-4aa7-ab28-04426045c82d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.182088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 15/33] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Aug 4 03:20:41 2022] POST /identity/v3/roles => generated 219 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:42.186648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.199359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e29f7-2302-4aa7-ab28-04426045c82d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.252488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7d81c-d529-4569-9adb-357934f54325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.253185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7d81c-d529-4569-9adb-357934f54325 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.253507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7d81c-d529-4569-9adb-357934f54325 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.253790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7d81c-d529-4569-9adb-357934f54325 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.254810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7d81c-d529-4569-9adb-357934f54325 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:42.256450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7d81c-d529-4569-9adb-357934f54325 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:42.319770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab973761-fca8-4802-92f2-766eee796827 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.362098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0bec9-f853-450b-ac23-8a043cc1d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.362766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0bec9-f853-450b-ac23-8a043cc1d18a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.363057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0bec9-f853-450b-ac23-8a043cc1d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.363337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0bec9-f853-450b-ac23-8a043cc1d18a None admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.365144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0bec9-f853-450b-ac23-8a043cc1d18a None admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:42.366837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0bec9-f853-450b-ac23-8a043cc1d18a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:42.368716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 16/34] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:42 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:42.369944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e62ce-6314-42ad-b7d2-ca785c6a054a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.370240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e62ce-6314-42ad-b7d2-ca785c6a054a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.370519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e62ce-6314-42ad-b7d2-ca785c6a054a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.390497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3b4e62ce-6314-42ad-b7d2-ca785c6a054a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:42.392010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3b4e62ce-6314-42ad-b7d2-ca785c6a054a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:42.417213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:42.463089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 17/35] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:42.464387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5325b53-e843-4b42-aad0-4d55e6d701fd None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.464683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5325b53-e843-4b42-aad0-4d55e6d701fd None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.465059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5325b53-e843-4b42-aad0-4d55e6d701fd None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.466343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 18/36] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:42.468251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5325b53-e843-4b42-aad0-4d55e6d701fd None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.474925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 19/37] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:20:42 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:42.476645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6f5ab-6b66-4828-9b59-1a862590d6eb None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.477116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6f5ab-6b66-4828-9b59-1a862590d6eb None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.477602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da6f5ab-6b66-4828-9b59-1a862590d6eb None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.479110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 20/38] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:42.480587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761a79-07a4-4721-9a43-219994746b13 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.481110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761a79-07a4-4721-9a43-219994746b13 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.481607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97761a79-07a4-4721-9a43-219994746b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.502515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-97761a79-07a4-4721-9a43-219994746b13 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:42.504114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-97761a79-07a4-4721-9a43-219994746b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:42.541241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:42.586835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 21/39] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:42.588141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.588434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.588716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.607336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5325b53-e843-4b42-aad0-4d55e6d701fd None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.611338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:42.612670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:42.647558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.648250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.648536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.648817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.650257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:42.652370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:42.655108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 19/40] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:42.655960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.686038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:42.696943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 22/41] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:42.697973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.763024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371f9e-ba86-45e3-86f7-0aa8cd0be505 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.796710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.796948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:42.797375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.797663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.797940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.798663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:42.800050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:42.801030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202e141-4efc-4905-80e1-f3e3486b9ebc None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.835074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.835716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.836050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.836523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.838083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:42.839675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:42.841286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 23/42] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:42.842026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.906436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 20/43] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Aug 4 03:20:42 2022] POST /identity/v3/roles => generated 225 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:42.907239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:42.954813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb15ad3-2af5-4bec-bfe6-b631db69755d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:42.995494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:42.996367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:42.996739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:42.997112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] PATH_INFO: `/v3/domains/default/users/4d0f702c6dd04ac1a8cf16b1ca420f19/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:42.999299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4d0f702c6dd04ac1a8cf16b1ca420f19, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:43.000860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:43.021624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:43.039201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2064-73fa-4827-a300-18fa1a1d9b27 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:43.077637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41989d0d-8bb9-4cab-8fd8-a23dd6ff4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:43.078444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41989d0d-8bb9-4cab-8fd8-a23dd6ff4332 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:43.078787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41989d0d-8bb9-4cab-8fd8-a23dd6ff4332 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:43.079119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41989d0d-8bb9-4cab-8fd8-a23dd6ff4332 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:43.079915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41989d0d-8bb9-4cab-8fd8-a23dd6ff4332 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:43.081939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41989d0d-8bb9-4cab-8fd8-a23dd6ff4332 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:43.100273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 24/44] 10.222.0.66 () {58 vars in 1361 bytes} [Thu Aug 4 03:20:42 2022] PUT /identity/v3/domains/default/users/4d0f702c6dd04ac1a8cf16b1ca420f19/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:20:43.101088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:43.138267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:43.179291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 21/45] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Aug 4 03:20:42 2022] POST /identity/v3/roles => generated 223 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:43.225744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147c678-43a6-44a5-a189-6b147da3e399 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:43.270387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed76373-cb03-4e61-8091-bdfd9b50dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:43.271067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed76373-cb03-4e61-8091-bdfd9b50dbad None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:43.271373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed76373-cb03-4e61-8091-bdfd9b50dbad None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:43.271697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed76373-cb03-4e61-8091-bdfd9b50dbad None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:43.272363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed76373-cb03-4e61-8091-bdfd9b50dbad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:43.273939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed76373-cb03-4e61-8091-bdfd9b50dbad None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:43.359437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 25/46] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:20:43 2022] POST /identity/v3/projects => generated 312 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:45.109374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30025f0e-f652-4de9-91ad-e0925c2c673f None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.109917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30025f0e-f652-4de9-91ad-e0925c2c673f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.109917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30025f0e-f652-4de9-91ad-e0925c2c673f None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.111292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 22/47] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:45.117063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829f685-39d3-4734-bfa2-892e455a6572 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.117492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829f685-39d3-4734-bfa2-892e455a6572 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.117667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829f685-39d3-4734-bfa2-892e455a6572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.136223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-5829f685-39d3-4734-bfa2-892e455a6572 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:45.137454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5829f685-39d3-4734-bfa2-892e455a6572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:45.210012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 26/48] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:20:45.232596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30025f0e-f652-4de9-91ad-e0925c2c673f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.334675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30025f0e-f652-4de9-91ad-e0925c2c673f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.340552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778d2bc-1829-4152-af08-78883473e74f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.340844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778d2bc-1829-4152-af08-78883473e74f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.341120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778d2bc-1829-4152-af08-78883473e74f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.342586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 27/49] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:20:45.347514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.347802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.348755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.367502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:45.368621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:45.369229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.369879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.370162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.370441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.370748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:45.382541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:45.388579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:45.389948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:45.390685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 23/50] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:20:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:45.396383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.400571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:45.437988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 28/51] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:20:45.461405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.494384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910b6d7-9007-483a-9c25-1aed3f84e758 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.527914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.529169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.529804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.530790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.531612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:45.531936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:45.533384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:45.545432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 24/52] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:45.550864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.561668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5732ca3-d194-41eb-8a79-13d66e7edb7b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.596194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.596840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.597126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.597438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.598823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:45.600303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:45.601946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 29/53] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:45.607606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.614933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:45.616272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:45.650997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f2bc-c594-405a-aa59-13549cd2df93 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.684125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.684770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.685058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.685390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.686567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:45.688038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:45.689784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 25/54] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:45 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:45.695284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.708893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee79e8-e5cf-41ba-a2e1-634b269bda5d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.743405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c63b3f-d5a8-46e5-8682-215e4919b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.744033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c63b3f-d5a8-46e5-8682-215e4919b5b5 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.744322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c63b3f-d5a8-46e5-8682-215e4919b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.744601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c63b3f-d5a8-46e5-8682-215e4919b5b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.745312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c63b3f-d5a8-46e5-8682-215e4919b5b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:45.746768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c63b3f-d5a8-46e5-8682-215e4919b5b5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:45.796842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ab09d-10bd-410b-8ef5-e2e46ade6bd9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.829767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.830399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.830687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.830964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.832126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:45.833915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:45.835499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 26/55] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:45.841338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:45.843927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:45.844033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 30/56] 10.222.0.66 () {60 vars in 1140 bytes} [Thu Aug 4 03:20:45 2022] POST /identity/v3/projects => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:20:45.949423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a59ec1-7ef9-4648-8862-f58a642f1495 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:45.985395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:45.986039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:45.986334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:45.986605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:45.987405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.domain.id=default, user.id=4d0f702c6dd04ac1a8cf16b1ca420f19) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:45.987720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:45.989844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdc1a3-763f-48d3-928d-178f2c32493a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:46.000145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 27/57] 10.222.0.66 () {56 vars in 1322 bytes} [Thu Aug 4 03:20:45 2022] GET /identity/v3/role_assignments?user.id=4d0f702c6dd04ac1a8cf16b1ca420f19&scope.domain.id=default&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:47.846667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee4fd-4300-4904-926d-76c295cc1fdd None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:47.847173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee4fd-4300-4904-926d-76c295cc1fdd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:47.847257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee4fd-4300-4904-926d-76c295cc1fdd None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:47.848689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 31/58] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:47.853943 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:47.854348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:47.854498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:47.872134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:47.873330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:47.930880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:47.937867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 28/59] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:47.962880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ee4fd-4300-4904-926d-76c295cc1fdd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:48.021490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:48.061634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ee4fd-4300-4904-926d-76c295cc1fdd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:48.096065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a394b-6137-4e62-8542-db7f576edab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:48.096750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a394b-6137-4e62-8542-db7f576edab1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:48.096987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a394b-6137-4e62-8542-db7f576edab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:48.097295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a394b-6137-4e62-8542-db7f576edab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:48.098621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a394b-6137-4e62-8542-db7f576edab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:48.100100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a394b-6137-4e62-8542-db7f576edab1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:48.101721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 32/60] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:48.107119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:48.202810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30784e8d-984d-4507-92c2-c9715a774878 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:48.235953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021570f-2a03-457b-8f52-4df8647821e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:48.236565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021570f-2a03-457b-8f52-4df8647821e6 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:48.236844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021570f-2a03-457b-8f52-4df8647821e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:48.237114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021570f-2a03-457b-8f52-4df8647821e6 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:48.237828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021570f-2a03-457b-8f52-4df8647821e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:48.239511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021570f-2a03-457b-8f52-4df8647821e6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:48.334787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 29/61] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:20:48 2022] POST /identity/v3/users => generated 275 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:51.389490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd1ff8-3bce-4c70-ace2-15e94cac43dc None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.390498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd1ff8-3bce-4c70-ace2-15e94cac43dc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.390993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd1ff8-3bce-4c70-ace2-15e94cac43dc None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.392595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 33/62] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:51.398424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.399236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.399720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.423038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:51.424570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:51.479703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a824670-aa4f-4e2e-ba54-dca770a750c6 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.480322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a824670-aa4f-4e2e-ba54-dca770a750c6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.480804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a824670-aa4f-4e2e-ba54-dca770a750c6 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.482401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 34/63] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:51.487353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.487862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.488340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.488751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:51.509880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 30/64] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:51.530213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:51.531430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:51.533350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:51.641730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 35/65] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:20:51.679035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:51.695165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28079b7d-f388-43d1-b795-ee50a5be21d1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:51.739831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb4e0c-ef99-4114-a22d-9997b986b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:51.740729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4e0c-ef99-4114-a22d-9997b986b892 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.741158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4e0c-ef99-4114-a22d-9997b986b892 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.741603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb4e0c-ef99-4114-a22d-9997b986b892 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.758375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb4e0c-ef99-4114-a22d-9997b986b892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-82bb4e0c-ef99-4114-a22d-9997b986b892 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:51.765031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:51.766060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:51.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:51.772849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 31/66] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:20:51.774848 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce5aa-16d1-4fa1-aeed-79614147ed1b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.775280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce5aa-16d1-4fa1-aeed-79614147ed1b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.775697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fce5aa-16d1-4fa1-aeed-79614147ed1b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.777143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 32/67] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:51.778036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce5aa-16d1-4fa1-aeed-79614147ed1b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:51.789806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:51.829013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:51.838808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b653-d766-4eb8-9ae1-13f290b9502d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:51.874735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e4033-e4a2-4a1c-9909-5d575b7d4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:51.875382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4033-e4a2-4a1c-9909-5d575b7d4d57 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.875669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4033-e4a2-4a1c-9909-5d575b7d4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.875951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e4033-e4a2-4a1c-9909-5d575b7d4d57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.886856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e4033-e4a2-4a1c-9909-5d575b7d4d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:51.891016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fce5aa-16d1-4fa1-aeed-79614147ed1b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b86e4033-e4a2-4a1c-9909-5d575b7d4d57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:51.891832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:51.892636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:51.893422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 36/68] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:20:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:20:51.894923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.894923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.895448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.912916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:51.914166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:51.934004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:51.934636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.934923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.935196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.935929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:51.936238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:51.937693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c3fc5e-4b51-412a-95f1-5f0f23df6ec1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:51.946577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 33/69] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:20:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:51.952440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb42f-d6f6-44b3-8293-352bb1194124 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:51.952735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb42f-d6f6-44b3-8293-352bb1194124 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:51.953020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaecb42f-d6f6-44b3-8293-352bb1194124 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:51.954323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 34/70] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:51.957498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb42f-d6f6-44b3-8293-352bb1194124 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:51.985347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 37/71] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:51.990212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.043519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.074160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaecb42f-d6f6-44b3-8293-352bb1194124 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.077557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.100507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16dd460-f601-49ca-9cdf-7dcd64d8219e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.119779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.120408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.120692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.121523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.122538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:52.122787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.124187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740181a1-e41f-4f31-934c-92c1521399fb None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.138209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 35/72] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:52.138209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.138209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.138209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.140252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.140910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.141217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.141511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.155397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:52.156593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:52.157145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.158657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.160405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 38/73] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:51 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:52.164077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.239244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 36/74] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:52.240015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.287287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16765adb-4003-4c7b-966a-d2a25703b476 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.289700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.322740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.323386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.323669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.323946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.335306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:52.340187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:52.341029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:52.342008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 39/75] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:52.342848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.343528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.349834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67815f72-bebb-4d24-b5cc-7bd2b7c13622 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.384065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.384712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.384999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.385301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.386564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.388044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.389883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 37/76] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:52.390481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.446868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d039bb-6254-43e4-bd57-3891646f782f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.481663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.482278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.482563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.482841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.490999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a38c7-1c87-49a9-8b06-4a859faae71e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.514613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.525570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49422e1-486f-430f-8ec9-8066e92a513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.526206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49422e1-486f-430f-8ec9-8066e92a513b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.526488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49422e1-486f-430f-8ec9-8066e92a513b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.526763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49422e1-486f-430f-8ec9-8066e92a513b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.536971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49422e1-486f-430f-8ec9-8066e92a513b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-e49422e1-486f-430f-8ec9-8066e92a513b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:52.541915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:52.542704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:52.543457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 38/77] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:52.544674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbc40-903b-4655-91bd-5658513aaeff None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.544964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbc40-903b-4655-91bd-5658513aaeff None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.545262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644bbc40-903b-4655-91bd-5658513aaeff None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.546509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 39/78] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:52.547260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbc40-903b-4655-91bd-5658513aaeff None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.557833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.559333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.572600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.650899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644bbc40-903b-4655-91bd-5658513aaeff None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.659459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 40/79] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:20:52.660287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.684428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.685052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.685379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.685662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.686385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:52.686696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.688098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.696730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 40/80] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:52.697546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.724312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.761840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e6753-a26a-42f2-aaea-55f5286a65aa None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.781603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.796526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16439f-c21a-4a42-ab2d-0aabaadb1fe9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:52.797879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.798505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.798791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.799100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.830289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:52.830901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.831184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.831458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.832169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:52.832998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.834423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be204af-8099-404f-823f-60cc3df37d17 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.842744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 41/81] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:52.843895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:52.844179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:52.844460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:52.862392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:52.863622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:52.867465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:52.868945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:52.929530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:52.930616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 42/82] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:52.931250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:52.976152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 41/83] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:20:52.977000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.026668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.049005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b4c06-0bed-49f3-aa7b-fd08278beab5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.078876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd87d14-3de2-4c01-9d8f-bc0b9318e8af None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.084820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.085498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.085823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.086212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.087398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.088896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.090615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 43/84] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.091377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.112962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.113757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.114053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.114331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.115500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.116976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.118626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 42/85] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:52 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.119380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.152698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.195711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918db546-768f-4d0b-93e8-14f0d4f5aafe None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.221321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190d25b-6892-4c5a-bd0a-2551ac6fc427 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.230116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.230763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.231046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.231323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.232525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.234041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.235651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 44/86] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.236408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.252515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.257611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.258255 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.258537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.258812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.260010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.261504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.263168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 43/87] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.263827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.336746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9c254-739f-4929-b187-c751664ea9cd None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.361438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.365580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253417e6-1fe2-4c7d-80e1-21a37dc92197 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.370954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.371591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.371871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.372145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.373326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.381802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.381802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 45/88] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.381802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.404376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.404997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.405312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.405597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.406380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=7c569f0a04f74c129f5270c857601f98) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:53.406692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.408161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.413627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 44/89] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/role_assignments?user.id=7c569f0a04f74c129f5270c857601f98&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.414413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.475292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.496871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a29cb-00be-4826-becb-c067e201aadc None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.535400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee024065-5bfa-4583-b274-4e36468a1819 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.537241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.538030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.538407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.538776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.575574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.576129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.576408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.576682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.609796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.617770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.623393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.650887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.652477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.704875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.728984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 46/90] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:20:53.728984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.768357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 45/91] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:20:53.769440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.836200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:53.838767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cded636-1529-4c6b-815b-8fdd5d719a99 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.876249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1e607-a99c-44f8-a5f5-e7cf4dc3bd36 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:53.877367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.878198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.878488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.880299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.880299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.881809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.912815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:53.913495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:53.913785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:53.914069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:53.914913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=4d0f702c6dd04ac1a8cf16b1ca420f19) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:53.915231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:53.916722 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:53.922141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 46/92] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/role_assignments?user.id=4d0f702c6dd04ac1a8cf16b1ca420f19&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:53.922914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:53.974032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 47/93] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:20:53 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:53.975470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:54.043868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:54.119253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aace6-0fb5-4dbc-baa3-7cdef6a4a1f5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:54.122875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42574bbd-8dab-47eb-9680-377dbe5a8279 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:54.175821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:54.177393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:54.178128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:54.178418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:54.178752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:54.180049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:54.181613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:54.183838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 48/94] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:54.185714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:54.186626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:54.187102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:54.187580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] PATH_INFO: `/v3/projects/59d7f197907a4687869fa37757e5b5ba` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:54.189461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] RBAC: Authorizing `identity:get_project(project_id=59d7f197907a4687869fa37757e5b5ba)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:54.191725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:54.191838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:54.193591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 47/95] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:53 2022] GET /identity/v3/projects/59d7f197907a4687869fa37757e5b5ba => generated 312 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:54.208194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:54.324486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f26ac-b578-4203-8355-703d43861c11 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:54.333972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:54.334136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a2fd4-0a9c-4e2e-aa59-ec2644964b35 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:54.369933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:54.370927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:54.371488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:54.371775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:54.372741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] RBAC: Adding query filter params (role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=7c569f0a04f74c129f5270c857601f98) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:54.373057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:54.375135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16431d54-a378-4a6c-94ee-1bec7bc74b8a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:54.380797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:54.381642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:54.381994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:54.382281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:54.382803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 49/96] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:20:54 2022] GET /identity/v3/role_assignments?user.id=7c569f0a04f74c129f5270c857601f98&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:54.383150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=59d7f197907a4687869fa37757e5b5ba, user.id=7c569f0a04f74c129f5270c857601f98) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:54.383513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:54.384989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1c5dc-351a-4eb1-87c8-762297deda3b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:54.391928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 48/97] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:20:54 2022] GET /identity/v3/role_assignments?user.id=7c569f0a04f74c129f5270c857601f98&scope.project.id=59d7f197907a4687869fa37757e5b5ba&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:55.752302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e2585-e186-4e6a-b987-647c4b5d9418 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:55.752830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e2585-e186-4e6a-b987-647c4b5d9418 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:55.752830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e2585-e186-4e6a-b987-647c4b5d9418 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:55.754894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 50/98] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:55.760224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:55.760662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:55.760840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:55.786697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:55.787916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:55.858685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:55.876640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 49/99] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:55.912102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e2585-e186-4e6a-b987-647c4b5d9418 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:56.048771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e2585-e186-4e6a-b987-647c4b5d9418 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:56.089112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:56.090139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:56.090619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:56.091095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:56.106922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:56.112561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:56.113423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:56.114322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:56.116572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 51/100] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:20:56.125095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:56.272426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437a250-1885-489f-9e9b-b58c7bef57d1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:56.314038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:56.314592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:56.314881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:56.315160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:56.315887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:56.316195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:56.318221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6d622-eaa6-45b3-b481-3395529252b4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:56.328178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 50/101] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:56.334700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:56.387705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:56.477676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ad7f9-0cf7-4c86-9a0b-e133331c61ce None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:56.517294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d098b8c-47ec-4232-9df1-aadf23621fe4 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:56.517586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d098b8c-47ec-4232-9df1-aadf23621fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:56.517866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d098b8c-47ec-4232-9df1-aadf23621fe4 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:56.519239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 51/102] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:56.527139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:56.527804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:56.528092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:56.528372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:56.529602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:56.531075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:56.532680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 52/103] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:56 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:56.536451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:56.536742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:56.537026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:56.540775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:56.579729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:56.580880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:56.716203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:56.733564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:56.740307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1c34d-01d8-4e70-b682-a0d2d889da12 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:56.751838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 52/104] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:56.787806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:56.827283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:56.827941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:56.828228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:56.828506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:56.833987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:56.835407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:56.837078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 53/105] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:56 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:56.844281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:56.953072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3d665-130c-424c-b99d-2bdef844c62e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.022935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.023964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.024476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.024950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.046987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:57.055026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:57.055922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:57.056654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 53/106] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:57.063518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.100394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff8c08-ca15-44c1-a835-2dde21e19943 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.155119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:57.193946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cba806-a743-45b5-82e2-786ab07b868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.194650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba806-a743-45b5-82e2-786ab07b868f None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.194934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba806-a743-45b5-82e2-786ab07b868f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.195212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cba806-a743-45b5-82e2-786ab07b868f None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/7c569f0a04f74c129f5270c857601f98/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.197328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cba806-a743-45b5-82e2-786ab07b868f None admin] RBAC: Authorizing `identity:create_grant(project_id=04d97b2c1dab478f84bc95fb26a2f107, user_id=7c569f0a04f74c129f5270c857601f98, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:57.198800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cba806-a743-45b5-82e2-786ab07b868f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:57.266926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec49c0d-c997-42e4-a447-bf14d4a135c7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.289347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 54/107] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:20:56 2022] PUT /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/7c569f0a04f74c129f5270c857601f98/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:20:57.290601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f45ead-6f42-425c-8c1a-3faa2b551829 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.290890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f45ead-6f42-425c-8c1a-3faa2b551829 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.291167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f45ead-6f42-425c-8c1a-3faa2b551829 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.292541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 55/108] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:57.297476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2c97f-766e-46f4-9e82-517acf8078dd None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.297685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2c97f-766e-46f4-9e82-517acf8078dd None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.297907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2c97f-766e-46f4-9e82-517acf8078dd None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.299367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 56/109] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:57.300336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.300617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.301462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.327580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:57.328832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:57.335807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.336460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.336743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.337019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.337805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:57.341528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:57.344803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a8693-e2a2-41f4-8187-e9b318fabbf6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:57.362896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 54/110] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:20:57.364520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.364812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.365091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.373017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:57.422100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:57.424376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:57.484444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 57/111] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:57.485139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.557943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 55/112] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:57.558924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.596760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:57.630596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8246c3-5f30-4e33-a863-ea6e7596d3ce None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.673256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.673601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba31b9-b3b0-4019-84e5-3b9ca857aacd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.673908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.674195 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.674466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.675740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:57.677503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:57.679138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 58/113] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:57.679899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.705452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:57.714715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.715372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.720061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.720061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.728134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:57.733422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:57.735479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:20:57.737360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:57.739025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 56/114] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:57.741229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.791113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff637bf-7058-43bf-b159-1941334edbe6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.836006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1ad0e-83da-43f3-955f-01a3d818ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.837587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1ad0e-83da-43f3-955f-01a3d818ebbc None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.838379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1ad0e-83da-43f3-955f-01a3d818ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.838794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1ad0e-83da-43f3-955f-01a3d818ebbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.847264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:57.855075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1ad0e-83da-43f3-955f-01a3d818ebbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:57.855684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52282ad-e83e-4c04-b4ed-20daa0897ea3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-eed1ad0e-83da-43f3-955f-01a3d818ebbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:57.869269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:57.870077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 59/115] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:57.870942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec49cf-166c-4204-b988-62e483566144 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.871603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec49cf-166c-4204-b988-62e483566144 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.871603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec49cf-166c-4204-b988-62e483566144 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.871603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 60/116] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:57.871603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec49cf-166c-4204-b988-62e483566144 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.904656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:57.905241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:57.905524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:57.905951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:57.907131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:57.908662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:57.910608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 57/117] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:57.911576 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:57.962973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:57.979608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec49cf-166c-4204-b988-62e483566144 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.017062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.017973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.018408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.018688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.019417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:58.020306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.021915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72bdff-293b-4d34-8b4b-00fc40cd847a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.022130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78b335-3c44-4198-9bbb-11aa9c0c02a8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.030493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 61/118] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.031677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.031965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.032242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.052524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:58.053767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:58.064898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.065878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.066169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.066473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.068165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:58.068901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.070787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b795e55-0be1-45aa-ac65-e5db487e7c2f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.085102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 58/119] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:20:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.085102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b70b5-03f6-43ba-b1dc-4cb164cc0bf7 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.085102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b70b5-03f6-43ba-b1dc-4cb164cc0bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.085102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b70b5-03f6-43ba-b1dc-4cb164cc0bf7 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.088461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 59/120] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:58.089162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b70b5-03f6-43ba-b1dc-4cb164cc0bf7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.116012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:58.127177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 62/121] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:58.141548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.206384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b70b5-03f6-43ba-b1dc-4cb164cc0bf7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.221042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:58.243615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247431dc-91a8-4205-8b22-19f8922f3842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.244303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247431dc-91a8-4205-8b22-19f8922f3842 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.244588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247431dc-91a8-4205-8b22-19f8922f3842 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.244866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247431dc-91a8-4205-8b22-19f8922f3842 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/4d0f702c6dd04ac1a8cf16b1ca420f19/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.246025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b56db3-4498-4776-9967-2d7fc30bd404 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.246964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247431dc-91a8-4205-8b22-19f8922f3842 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d97b2c1dab478f84bc95fb26a2f107, user_id=4d0f702c6dd04ac1a8cf16b1ca420f19, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.248928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247431dc-91a8-4205-8b22-19f8922f3842 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.280903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.281606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.281882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.282154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.283317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.284776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.287079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 63/122] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.291710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.345478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 60/123] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:20:58 2022] PUT /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/4d0f702c6dd04ac1a8cf16b1ca420f19/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:20:58.346871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.347168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.347442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.368603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:58.369916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:58.415626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9d937-3af5-47cd-b530-7967d19a2753 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.491382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 61/124] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:20:58.491558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.493431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.493722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.494001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.495401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.495973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.497497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.499284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 64/125] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.499925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.515158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:58.526335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:58.622317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7e79-a5ac-48df-80b1-dde43e69772b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.622851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44728df8-15ab-4ace-a29e-4aeca291be6f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.665428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.666089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.666445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.667025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.672266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.673402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.673877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.674386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] PATH_INFO: `/v3/projects/59d7f197907a4687869fa37757e5b5ba` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.676890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] RBAC: Authorizing `identity:get_project(project_id=59d7f197907a4687869fa37757e5b5ba)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.679045 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:20:58.679479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.681525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 65/126] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/projects/59d7f197907a4687869fa37757e5b5ba => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.682755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:20:58.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:20:58.685185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 62/127] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:20:58.686926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.792094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:58.795260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608587a-8e0c-4a6a-812a-9cc601c24259 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.817907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:58.822309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad9fe7c-af56-4e78-b6cb-eac6c6562217 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:58.834547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.835520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.835977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.837150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.839983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.842291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.844437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 66/128] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.847628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.868815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:58.869514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:58.869793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:58.870073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:58.871880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:58.873386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:58.877299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 63/129] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:58.878246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:58.984715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e534d-da43-4ea2-a844-a15ffd10e14b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:59.002985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f031a-ff9b-41a4-92a2-f58d2d57ebed None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:59.023603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:59.024283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.024682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.024968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] PATH_INFO: `/v3/projects/59d7f197907a4687869fa37757e5b5ba/users/7c569f0a04f74c129f5270c857601f98/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.027943 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=59d7f197907a4687869fa37757e5b5ba, user_id=7c569f0a04f74c129f5270c857601f98, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:59.030109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:59.041210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:59.041971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.042413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.042703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.044002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:20:59.044548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:59.047144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:59.059442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 64/130] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:20:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:59.060060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:59.066449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:59.118958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 67/131] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:20:58 2022] PUT /identity/v3/projects/59d7f197907a4687869fa37757e5b5ba/users/7c569f0a04f74c129f5270c857601f98/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:20:59.120662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:59.139476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:59.213860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc936c67-21f1-466a-a3a8-e514ceba14cc None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:59.273152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:59.274328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.274811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.275312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/7c569f0a04f74c129f5270c857601f98/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.277804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] RBAC: Authorizing `identity:create_grant(project_id=04d97b2c1dab478f84bc95fb26a2f107, user_id=7c569f0a04f74c129f5270c857601f98, role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:59.279600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:59.318134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcfbfb-4c4f-4b64-82ee-76b6994795d8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:59.353084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 65/132] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:20:59 2022] PUT /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/7c569f0a04f74c129f5270c857601f98/roles/7feebec2e0484087af48f1cb50e6f954 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:20:59.354403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:59.357219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:59.357219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.357219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.357219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.357909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:59.359400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:59.458808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:59.484751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 68/133] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:20:59 2022] POST /identity/v3/projects => generated 302 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:20:59.533756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aaddc7-8019-4c03-8031-6c7b3e35d61f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:59.575583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b923-4bff-4945-850e-8005dbb45f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:59.577534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b923-4bff-4945-850e-8005dbb45f3e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.577826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b923-4bff-4945-850e-8005dbb45f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.578114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b923-4bff-4945-850e-8005dbb45f3e None admin] PATH_INFO: `/v3/users/51ff4b95aff74d17bd71f44e8e9320d2` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.600193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b923-4bff-4945-850e-8005dbb45f3e None admin] RBAC: Authorizing `identity:get_user(user_id=51ff4b95aff74d17bd71f44e8e9320d2)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:59.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b923-4bff-4945-850e-8005dbb45f3e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:59.603869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 66/134] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:20:59 2022] GET /identity/v3/users/51ff4b95aff74d17bd71f44e8e9320d2 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:20:59.609582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:20:59.698425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:20:59.747819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e484dd-f7ee-4d65-91f4-ad7701bae7d1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:20:59.750458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a7c5-c9cf-4c9a-9863-863707188bd9 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.750754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a7c5-c9cf-4c9a-9863-863707188bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.751042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a7c5-c9cf-4c9a-9863-863707188bd9 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.752592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 67/135] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:20:59.760894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.761187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.761498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.795275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:20:59.796439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:20:59.801892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:20:59.802810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:20:59.804102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:20:59.804587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:20:59.928795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 68/136] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:20:59.936623 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:20:59.938201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:20:59.970244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:00.014002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:00.072830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 69/137] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:20:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:00.078629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:00.097058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:00.203474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b83f5-1eed-4f0e-b431-a96dca0e902f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:00.264261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:00.264925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.265225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.265507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.297367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:00.302442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:00.303271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:00.303995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 69/138] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:20:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:00.311031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:00.367363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b06fd-dddb-41ed-8be4-890867dc697c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:00.461944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:00.462530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.462815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.463094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.464406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:00.465901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:00.477468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 70/139] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:00 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:00.478434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:00.528353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:00.534658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280acf7-9e18-44ca-acb7-f29bdc606a84 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:00.558665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:00.595921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:00.596553 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.596840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.597121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.598475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:00.598884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:00.600296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:00.620380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 70/140] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:00.627408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:00.668257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9c048-b55b-47d6-a427-1aa795b22d50 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:00.741802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:00.742365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.742651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.742931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.743751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] RBAC: Adding query filter params (role.id=7459c8cfcd03449b9c47588138df4243, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=51ff4b95aff74d17bd71f44e8e9320d2) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:00.744065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:00.745595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:00.761845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 71/141] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:00 2022] GET /identity/v3/role_assignments?user.id=51ff4b95aff74d17bd71f44e8e9320d2&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=7459c8cfcd03449b9c47588138df4243 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:00.810152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73768-0e15-4811-bd0f-8ecde3c93584 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:00.848523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c234f5-0cd9-42c1-8688-8f08e88e7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:00.849189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c234f5-0cd9-42c1-8688-8f08e88e7cef None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.849508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c234f5-0cd9-42c1-8688-8f08e88e7cef None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.849793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c234f5-0cd9-42c1-8688-8f08e88e7cef None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.851096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c234f5-0cd9-42c1-8688-8f08e88e7cef None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:00.852614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c234f5-0cd9-42c1-8688-8f08e88e7cef None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:00.854527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 71/142] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:00 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:00.859930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:00.924690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7f71d-205b-4521-a437-8a372d6a7095 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.925056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7f71d-205b-4521-a437-8a372d6a7095 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.925275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7f71d-205b-4521-a437-8a372d6a7095 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.927197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 72/143] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:00.934977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:00.935564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:00.935564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:00.969702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:00.970928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:01.018641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:01.036235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83583f-d686-485e-8727-ade9f7e5ac34 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:01.067631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:01.097828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 73/144] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:01.118838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:01.119497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:01.119863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:01.120136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:01.121223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:01.233995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d514a-d01a-481b-86cd-ef94dac85ffd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:01.274544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:01.276153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:01.277224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:01.277799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:01.292928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:01.294647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:01.300418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:01.306906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:01.307931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:01.308870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:01.316252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 74/145] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:01.327128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:01.388762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:01.482837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:01.495304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 72/146] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:01.501904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:01.533167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31ee24-4fcc-4969-975d-bb359c564c60 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:01.600256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:01.601780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:01.602417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:01.603384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:01.604405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:01.607019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:01.608563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:01.626313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 75/147] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:01.635403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:01.682409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1946f48-2398-4572-9859-a4ffdb1b5e86 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:01.735636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:01.736438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:01.736818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:01.737225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:01.739148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:01.740757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:01.742876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 73/148] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:01 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:01.759850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:01.778047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:01.786145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94281c8-a0af-4c7b-bf4c-ebdeff86facf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:01.826745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:01.829034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:01.829567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:01.831334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] PATH_INFO: `/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:01.834709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] RBAC: Authorizing `identity:get_user(user_id=4d0f702c6dd04ac1a8cf16b1ca420f19)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:01.837228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:01.839398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 76/149] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:01 2022] GET /identity/v3/users/4d0f702c6dd04ac1a8cf16b1ca420f19 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:01.846300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:01.861933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:01.938652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de83228-318a-4c24-ae38-838e642fdef7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:01.999102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f9ae6-858d-4d4d-956b-66b78268f414 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.039320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.040583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.041458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.042350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.042856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.043327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.043793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.044256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.045321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=7c569f0a04f74c129f5270c857601f98) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:02.046412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:02.048428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1c4ce-121f-4930-a76b-3e07e7a6ab43 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:02.065732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 74/150] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:01 2022] GET /identity/v3/role_assignments?user.id=7c569f0a04f74c129f5270c857601f98&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:02.067618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530fce-62d4-4c9b-837a-d9452509d57a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.074420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530fce-62d4-4c9b-837a-d9452509d57a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.075085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530fce-62d4-4c9b-837a-d9452509d57a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.078444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 75/151] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:02.084204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebc157-9650-49c2-8746-dbad5cebcdc2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.084811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebc157-9650-49c2-8746-dbad5cebcdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.085437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebc157-9650-49c2-8746-dbad5cebcdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.111073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.122401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-03ebc157-9650-49c2-8746-dbad5cebcdc2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:02.123946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-03ebc157-9650-49c2-8746-dbad5cebcdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:02.172235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:02.173770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:02.293368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 76/152] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:02.294719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabc53-325e-4db8-9c07-3e978a8245a6 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.295006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabc53-325e-4db8-9c07-3e978a8245a6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.295379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfabc53-325e-4db8-9c07-3e978a8245a6 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.296668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 77/153] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:02.303121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.303407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.303680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.315387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 77/154] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:02.321568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.339106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:02.341826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:02.401787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.412408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.421508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 78/155] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:02.422703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.469418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de410-af6a-42be-993e-16bffe805b41 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.506915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.507567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.507924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.508206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.509426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:02.510896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:02.512696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 78/156] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:02.513556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.546860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a8354-c5d2-412e-8c4f-75ea3ae933c1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.585712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.586367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.586651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.586927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.598804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:02.603809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:02.605810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 79/157] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:02.607727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.622938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df1864-74bb-418d-be27-e553e58bfd1c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.667045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.668414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.679409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.692140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.692140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.692140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.696105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:02.702301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:02.703276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:02.704014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:02.706862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 79/158] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:02.713155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.744021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be064e-0859-49d1-a8de-3ee294c593a1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.782754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.783372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.783650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.783920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.784710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=4d0f702c6dd04ac1a8cf16b1ca420f19) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:02.785019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:02.787051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:02.793016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 80/159] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/role_assignments?user.id=4d0f702c6dd04ac1a8cf16b1ca420f19&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:02.793813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.847602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375fa20-f978-4b9a-ae13-874f887d3536 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.894135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.895369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.895643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.895918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.896645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:02.896946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:02.898423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58e43b-acc2-406f-b47e-5fce8c71db8e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:02.908182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 80/160] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:02.909498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0b471-71b4-4d2e-ac3f-966ba3a0625b None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.909782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0b471-71b4-4d2e-ac3f-966ba3a0625b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.910157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0b471-71b4-4d2e-ac3f-966ba3a0625b None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.911757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 81/161] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:02.916629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0b471-71b4-4d2e-ac3f-966ba3a0625b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:02.924942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0cc13-d9f7-4111-b593-f73c914f2e7a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:02.936057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.969856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:02.971105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.971594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.972363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:02.973898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:02.974406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:02.976131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978f42e-40e7-453f-b8e8-a6c65c59752c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:02.985224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 81/162] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:02.985354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:02.986876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:02.986983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:02.987613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.006422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:03.007871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:03.056476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0b471-71b4-4d2e-ac3f-966ba3a0625b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.097191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 82/163] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:03.098419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.110054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.111720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.112231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.112685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.115534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.117903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.120237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 82/164] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:02 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:03.126285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.214281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.217121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7ef7-ac98-4f6d-9960-aa6dd0a113e6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.244815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c6852-bba1-4a8b-88b6-f4564a69de72 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.252333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.253093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.253415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.253690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] PATH_INFO: `/v3/users/7c569f0a04f74c129f5270c857601f98` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.255549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] RBAC: Authorizing `identity:get_user(user_id=7c569f0a04f74c129f5270c857601f98)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.257001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.258640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 83/165] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/users/7c569f0a04f74c129f5270c857601f98 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:03.263147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.276593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.290481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.291104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.291409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.291663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.368611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.369595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55641cb-7fc0-49dc-9e3d-3f5b7a10aeb6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.370630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.406941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.408059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.408059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.408226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.409322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.410905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.412754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 84/166] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:03.413399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.451338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.481640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 83/167] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 03:21:03.482753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.506136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.528253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfb1cb-68f3-43b8-8360-e58f0f29b2d4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.564133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.564768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.565046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.565345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.591082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c9932-1579-4fd7-97e0-72d2f422c986 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.627418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf3342-a68f-4aa4-8439-3eb2830bc53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.628261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf3342-a68f-4aa4-8439-3eb2830bc53a None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.628552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf3342-a68f-4aa4-8439-3eb2830bc53a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.629622 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf3342-a68f-4aa4-8439-3eb2830bc53a None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.630457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf3342-a68f-4aa4-8439-3eb2830bc53a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.631934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf3342-a68f-4aa4-8439-3eb2830bc53a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.638040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.639516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.672794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.734944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 85/168] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:03.735816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.744485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 84/169] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:03 2022] POST /identity/v3/users => generated 283 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:03.745815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e6502-777b-42eb-9575-e2acab42f923 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.746110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e6502-777b-42eb-9575-e2acab42f923 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.746380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e6502-777b-42eb-9575-e2acab42f923 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.747665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 85/170] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:03.748387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e6502-777b-42eb-9575-e2acab42f923 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.842106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.856485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0f801-9f8f-43dd-8155-a59e7866a115 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.869373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e6502-777b-42eb-9575-e2acab42f923 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:03.897040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5017bf1-2324-456f-a2ea-d6bcffd3b79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.897649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5017bf1-2324-456f-a2ea-d6bcffd3b79d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.897932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5017bf1-2324-456f-a2ea-d6bcffd3b79d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.898203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5017bf1-2324-456f-a2ea-d6bcffd3b79d None admin] PATH_INFO: `/v3/projects/59d7f197907a4687869fa37757e5b5ba` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.899473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5017bf1-2324-456f-a2ea-d6bcffd3b79d None admin] RBAC: Authorizing `identity:get_project(project_id=59d7f197907a4687869fa37757e5b5ba)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.900980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5017bf1-2324-456f-a2ea-d6bcffd3b79d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.902690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 86/171] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/projects/59d7f197907a4687869fa37757e5b5ba => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:03.903890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.904182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.904459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.911197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:03.912149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:03.912429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:03.912700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:03.913957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:03.914455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:03.916224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:03.918143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 86/172] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:03.919297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:03.926350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:03.928568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:04.011240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 87/173] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:04.012410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:04.057935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d865f-ec99-4283-a6d5-af191ccc4e4f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:04.110573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:04.111370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:04.112545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:04.112912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:04.114520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=59d7f197907a4687869fa37757e5b5ba, user.id=7c569f0a04f74c129f5270c857601f98) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:04.114813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:04.116364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:04.123296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 87/174] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:03 2022] GET /identity/v3/role_assignments?user.id=7c569f0a04f74c129f5270c857601f98&scope.project.id=59d7f197907a4687869fa37757e5b5ba&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:04.124323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:04.137472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:04.145719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27dfe-6690-4cd6-abfc-84ef7309b0bd None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:04.211069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:04.211713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:04.211989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:04.212288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:04.213858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] RBAC: Adding query filter params (role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=7c569f0a04f74c129f5270c857601f98) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:04.214175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:04.216025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:04.222177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:04.227974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 88/175] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:04 2022] GET /identity/v3/role_assignments?user.id=7c569f0a04f74c129f5270c857601f98&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:04.266861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd7973-03a1-488a-a226-5c85cfdccdf5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:04.309263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:04.310188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:04.310616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:04.311037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:04.325236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:04.331262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:04.332289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:04.333049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 88/176] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:04.340871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:04.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d7f76-6fa8-425e-837b-dff7f731732c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:04.525689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:04.526315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:04.526592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:04.526863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:04.527579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:04.527878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:04.529311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:04.539865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 89/177] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:04.545635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:04.622519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:04.661292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83af6f8e-6b3f-4956-843c-5263f7c7b790 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:04.699855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:04.700501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:04.700776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:04.701047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] PATH_INFO: `/v3/users/51ff4b95aff74d17bd71f44e8e9320d2` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:04.702256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] RBAC: Authorizing `identity:get_user(user_id=51ff4b95aff74d17bd71f44e8e9320d2)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:04.703744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:04.705401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 89/178] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:04 2022] GET /identity/v3/users/51ff4b95aff74d17bd71f44e8e9320d2 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:04.711772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:04.776993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:04.821942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a71a-4261-4da8-be99-857270c0ff5e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:04.858079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48b6da-c20a-4de2-b25f-2d1639ec6e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:04.858722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48b6da-c20a-4de2-b25f-2d1639ec6e36 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:04.859048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48b6da-c20a-4de2-b25f-2d1639ec6e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:04.859317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48b6da-c20a-4de2-b25f-2d1639ec6e36 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:04.860482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48b6da-c20a-4de2-b25f-2d1639ec6e36 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:04.862026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48b6da-c20a-4de2-b25f-2d1639ec6e36 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:04.868873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 90/179] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:04 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:04.870194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:04.978114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937e45d-0f03-4099-90ea-16e77f19ffd2 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:05.013994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feda77-ad52-40b9-883a-290c29d081ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:05.015043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feda77-ad52-40b9-883a-290c29d081ef None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:05.015531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feda77-ad52-40b9-883a-290c29d081ef None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:05.016029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feda77-ad52-40b9-883a-290c29d081ef None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/51ff4b95aff74d17bd71f44e8e9320d2/roles/7459c8cfcd03449b9c47588138df4243` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:05.018240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feda77-ad52-40b9-883a-290c29d081ef None admin] RBAC: Authorizing `identity:create_grant(project_id=04d97b2c1dab478f84bc95fb26a2f107, user_id=51ff4b95aff74d17bd71f44e8e9320d2, role_id=7459c8cfcd03449b9c47588138df4243)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:05.020388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feda77-ad52-40b9-883a-290c29d081ef None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:05.024176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:05.112174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 90/180] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:04 2022] PUT /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/users/51ff4b95aff74d17bd71f44e8e9320d2/roles/7459c8cfcd03449b9c47588138df4243 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:06.157127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526fb02-8480-4358-ae30-6f6e3b3e9e13 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.157647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526fb02-8480-4358-ae30-6f6e3b3e9e13 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.157719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526fb02-8480-4358-ae30-6f6e3b3e9e13 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.159171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 91/181] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:06.165100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1181c-8951-42aa-bb37-45a52b36b676 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.165991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1181c-8951-42aa-bb37-45a52b36b676 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.166471 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1181c-8951-42aa-bb37-45a52b36b676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.186980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-fee1181c-8951-42aa-bb37-45a52b36b676 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:06.188200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fee1181c-8951-42aa-bb37-45a52b36b676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:06.261318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 91/182] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:06.284648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526fb02-8480-4358-ae30-6f6e3b3e9e13 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.330959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0658e-9cff-4cf0-8389-b8c782130f18 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.331244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0658e-9cff-4cf0-8389-b8c782130f18 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.331556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0658e-9cff-4cf0-8389-b8c782130f18 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.333599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 92/183] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:06.339113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e59de-cadf-4c31-9ba5-f849f16a179e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.339572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e59de-cadf-4c31-9ba5-f849f16a179e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.339892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e59de-cadf-4c31-9ba5-f849f16a179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.361663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-fc7e59de-cadf-4c31-9ba5-f849f16a179e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:06.362853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fc7e59de-cadf-4c31-9ba5-f849f16a179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:06.396563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526fb02-8480-4358-ae30-6f6e3b3e9e13 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:06.424475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:06.433618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:06.434263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.437335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.438948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 93/184] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:06.439049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.439121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419d4ca-65d2-4348-9592-ae4c5463ed05 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.439121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419d4ca-65d2-4348-9592-ae4c5463ed05 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.439121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419d4ca-65d2-4348-9592-ae4c5463ed05 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.441379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 94/185] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:06.443903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.444184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.444478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.453649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:06.459990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:06.460863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:06.461685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:06.462955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 92/186] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:06.463322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:06.463417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.464540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:06.518789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:06.541247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 95/187] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:06.541247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.571924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f70021-7bc4-484a-bbf2-5fd34730e5ea None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:06.607014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:06.607662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.607942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.608223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.609390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:06.610851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:06.617228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 93/188] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:06.617228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.663269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc474-6e3e-4d5a-ad00-aeeb0a3b7a24 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:06.677693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:06.700579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:06.701216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.701523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.701799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.702523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:06.702825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:06.704224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:06.713451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 96/189] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:06.713451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.729584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc254d4d-43dc-4af4-80f9-aab3471d810c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:06.764249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:06.764887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.765166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.765466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.769122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:06.776480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:06.782022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:06.782877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:06.783642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:06.788061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 94/190] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:06.788907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.825593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c6ad9-1d03-4581-9996-db0f3836f638 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:06.862920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:06.864089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.864346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.864616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.865334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:06.866792 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:06.899148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16642789-f31b-49ee-a070-43770cd2ee3b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:06.915103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:06.932947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:06.933552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:06.933833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:06.934101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] PATH_INFO: `/v3/users/ef3e40e556684481b491af215b7f49b9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:06.950518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] RBAC: Authorizing `identity:get_user(user_id=ef3e40e556684481b491af215b7f49b9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:06.951990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:06.953660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 95/191] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/users/ef3e40e556684481b491af215b7f49b9 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:06.959055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:06.976156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 97/192] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:06 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:06.976156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.083112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53b225-89ef-4ac3-972c-9633d8182dce None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.119022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722624a-b0a0-4ce8-a564-0b99b88a0d96 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.121869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:07.139469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.140104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.140386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.140663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.141421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:07.141730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:07.143120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:07.153466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 96/193] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:07.159561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.166473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df120a35-f105-4fcc-a8ff-e91c6f7b38d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.167739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120a35-f105-4fcc-a8ff-e91c6f7b38d6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.168197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120a35-f105-4fcc-a8ff-e91c6f7b38d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.168639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120a35-f105-4fcc-a8ff-e91c6f7b38d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.207882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:07.256862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df120a35-f105-4fcc-a8ff-e91c6f7b38d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:07.258199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df120a35-f105-4fcc-a8ff-e91c6f7b38d6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:07.289588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7780314-3f5f-4571-94cb-9fa0d4cdd4a5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.336266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd97668-92d6-4f63-8190-2d5d4608c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.337067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd97668-92d6-4f63-8190-2d5d4608c755 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.337428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd97668-92d6-4f63-8190-2d5d4608c755 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.337698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd97668-92d6-4f63-8190-2d5d4608c755 None admin] PATH_INFO: `/v3/users/ef3e40e556684481b491af215b7f49b9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.338868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd97668-92d6-4f63-8190-2d5d4608c755 None admin] RBAC: Authorizing `identity:get_user(user_id=ef3e40e556684481b491af215b7f49b9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:07.340318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd97668-92d6-4f63-8190-2d5d4608c755 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:07.345420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 97/194] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/users/ef3e40e556684481b491af215b7f49b9 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:07.345420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30687b-da4e-4e56-b2a4-546b0e105bd1 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.345420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30687b-da4e-4e56-b2a4-546b0e105bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.345420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30687b-da4e-4e56-b2a4-546b0e105bd1 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.345420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 98/195] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:07.350260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30687b-da4e-4e56-b2a4-546b0e105bd1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.400316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 98/196] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:07.401594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.401878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.402149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.409619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:07.421320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:07.422584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:07.489333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30687b-da4e-4e56-b2a4-546b0e105bd1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.489333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:07.512104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.513018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.513497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.513930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.514046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 99/197] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:07.520460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.619270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:07.620769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:07.645975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce6e2-a371-40f3-bb6d-d4b60b94c206 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.683807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.684462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.684885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.685315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.686641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:07.688803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:07.691000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 100/198] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:07.696666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.705512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:07.737469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 99/199] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:07.739571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.743163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:07.820694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d6cba-757d-483f-8a0b-73e14f94ea54 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.861323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b7c06-3144-496a-a5f0-7130746c37ad None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:07.865217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.866075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.866508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.866928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.878358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:07.885227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:07.886312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:07.887200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:07.889159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 101/200] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:07.890089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.905307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:07.905941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:07.906221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:07.906504 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:07.907576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=ef3e40e556684481b491af215b7f49b9) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:07.907893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:07.909496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:07.916601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 100/201] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/role_assignments?user.id=ef3e40e556684481b491af215b7f49b9&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:07.917142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:07.962955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:08.016000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:08.038180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514becf-dc1b-4e2a-b8b2-00266a5cf47e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:08.092457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc6187-250e-47ed-a7f2-f147f9133356 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:08.120728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:08.122482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:08.123181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:08.124415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:08.126932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:08.130587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:08.135090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 102/202] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:08.143211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:08.155535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:08.156433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:08.156713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:08.156985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:08.157746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:08.158049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:08.159862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:08.177685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 101/203] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:08.188722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:08.334900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b326c1-a32f-4386-8d9b-b25685e25fe0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:08.378040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:08.383057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:08.383723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:08.384011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:08.384291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] PATH_INFO: `/v3/users/51ff4b95aff74d17bd71f44e8e9320d2` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:08.385532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] RBAC: Authorizing `identity:get_user(user_id=51ff4b95aff74d17bd71f44e8e9320d2)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:08.387608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:08.389400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 102/204] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:08 2022] GET /identity/v3/users/51ff4b95aff74d17bd71f44e8e9320d2 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:08.395790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:08.396668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c352f8-d2c4-4e22-991f-2c3623b6f618 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:08.457483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:08.469740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:08.470434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:08.470822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:08.471101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:08.472028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] RBAC: Adding query filter params (role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=ef3e40e556684481b491af215b7f49b9) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:08.472340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:08.474586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:08.487856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 103/205] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:08 2022] GET /identity/v3/role_assignments?user.id=ef3e40e556684481b491af215b7f49b9&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 259 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:08.517774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a853f3-d38f-41ad-b134-f60435a5b466 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:08.555370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:08.556014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:08.556298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:08.556575 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:08.628938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:08.630751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:08.639530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:08.738717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 103/206] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:08.746491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:08.871400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73554f-5134-4dba-be2d-3f8ee2c512f1 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:08.919351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be2fa5d-41f2-4308-97f0-cf82e4167aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:08.920354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2fa5d-41f2-4308-97f0-cf82e4167aac None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:08.920832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2fa5d-41f2-4308-97f0-cf82e4167aac None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:08.921345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be2fa5d-41f2-4308-97f0-cf82e4167aac None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:08.922734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2fa5d-41f2-4308-97f0-cf82e4167aac None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:08.924381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be2fa5d-41f2-4308-97f0-cf82e4167aac None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:08.926545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 104/207] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:08 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:08.932589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:09.036949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:09.044972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83d59-f8fb-4e1a-b3f8-bbcbb1c443dd None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:09.084817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:09.085469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.085754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.086026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.086830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] RBAC: Adding query filter params (role.id=7459c8cfcd03449b9c47588138df4243, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107, user.id=51ff4b95aff74d17bd71f44e8e9320d2) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:09.087231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:09.088781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b770f-d2dd-4d6f-83bc-ddc13dc81d48 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:09.095212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 104/208] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:08 2022] GET /identity/v3/role_assignments?user.id=51ff4b95aff74d17bd71f44e8e9320d2&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=7459c8cfcd03449b9c47588138df4243 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:09.357861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd980fb-6d57-4c6e-b880-e2e130e0ad43 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.359866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd980fb-6d57-4c6e-b880-e2e130e0ad43 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.359866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd980fb-6d57-4c6e-b880-e2e130e0ad43 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.360192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 105/209] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:09.365153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eaf10-c28b-491e-9dc0-1e61421dcde1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.365649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eaf10-c28b-491e-9dc0-1e61421dcde1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.365742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046eaf10-c28b-491e-9dc0-1e61421dcde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.385662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-046eaf10-c28b-491e-9dc0-1e61421dcde1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:09.386921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-046eaf10-c28b-491e-9dc0-1e61421dcde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:09.458408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 105/210] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:09.483768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd980fb-6d57-4c6e-b880-e2e130e0ad43 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:09.551459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:09.593441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd980fb-6d57-4c6e-b880-e2e130e0ad43 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:09.614680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c75d9a-c68f-47eb-a2d2-34a69a20881d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.615021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c75d9a-c68f-47eb-a2d2-34a69a20881d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.615226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c75d9a-c68f-47eb-a2d2-34a69a20881d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.616590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 106/211] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:09.621674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.621954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.622225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.635094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:09.635743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.636025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.636755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.646256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:09.647727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:09.648334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:09.654801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:09.655692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:09.656442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 106/212] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:09.664990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:09.740805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 107/213] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:09.767006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:09.783629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a16330-fa62-4079-9d26-45078db8ebc3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:09.802059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:09.813434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:09.828788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:09.829817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.830172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.830518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.831436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:09.832898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:09.834916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:09.847089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 107/214] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:09.852585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:09.909845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1522d63-ca11-459a-9193-fedf97349585 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:09.955200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:09.955913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:09.956192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:09.956491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:09.957823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:09.959313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:09.961113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 108/215] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:09.966433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:09.978783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013baf4-d547-4293-a93e-dc94f24f52f1 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.030754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.032166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.032881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.033369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] PATH_INFO: `/v3/users/93a509336b6c431ca0376c0bb80364dd` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.053620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] RBAC: Authorizing `identity:get_user(user_id=93a509336b6c431ca0376c0bb80364dd)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:10.055482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:10.061993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 108/216] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:09 2022] GET /identity/v3/users/93a509336b6c431ca0376c0bb80364dd => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:10.063204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:10.070187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.079882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b926ec-d305-449d-8e0e-c8e4e7dd5a80 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.114630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.115989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b86e1-8a36-458c-bccc-562512f017e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.116660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b86e1-8a36-458c-bccc-562512f017e7 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.116941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b86e1-8a36-458c-bccc-562512f017e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.117301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b86e1-8a36-458c-bccc-562512f017e7 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.117951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b86e1-8a36-458c-bccc-562512f017e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:10.119398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b86e1-8a36-458c-bccc-562512f017e7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:10.183021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e273-d2d6-436a-964a-983c3bf5b762 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.205497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 109/217] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:09 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:21:10.229801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.230776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.231197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.231614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.286389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec321d-5bc0-442c-9831-cc01c14f2df5 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.286389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec321d-5bc0-442c-9831-cc01c14f2df5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.286792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec321d-5bc0-442c-9831-cc01c14f2df5 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.288586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 110/218] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:10.295090 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.295489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.296138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.322384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:10.322703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:10.324045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:10.324463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:10.375755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.415775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 111/219] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:10.437833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:10.463645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 109/220] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:10.469408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:10.541463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.583736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08289596-0c94-4519-97bf-f50bc0252c62 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.596761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cc375-f9ad-454a-8424-4efd701448c6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.640557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.641435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.641724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.641993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.643687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.644896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.645319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.645599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.646809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:10.648297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:10.650481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 110/221] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:10 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:10.654873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:10.656616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:10.661920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:10.662804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:10.663587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 112/222] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:10.670206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:10.674245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.780492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34da0-5b3d-4839-9a0f-5730db3836c2 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.820668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.821683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.822929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.823933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.825038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=93a509336b6c431ca0376c0bb80364dd) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:10.827041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dec31-1257-49bd-8a31-afeff0828e21 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:10.827510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:10.830638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:10.843123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 111/223] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:10 2022] GET /identity/v3/role_assignments?user.id=93a509336b6c431ca0376c0bb80364dd&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:10.873727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:10.874721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.875161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.875575 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.876491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:10.876944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:10.878562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcc02f8-b6c7-425e-accc-7f4c3b50df57 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:10.882420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.890314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 113/224] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:10.896459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:10.983825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:10.986487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4dcc2-d0d2-492f-a46d-b6bd62281a34 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:10.987066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4dcc2-d0d2-492f-a46d-b6bd62281a34 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:10.987541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4dcc2-d0d2-492f-a46d-b6bd62281a34 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:10.990797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 114/225] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:11.000043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.000272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.000549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.020171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:11.021303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:11.033247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c84bd6-ca9d-4071-b856-2aa7145ad9e0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:11.102452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:11.103039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.103323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.103732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 115/226] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:11.103853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] PATH_INFO: `/v3/users/ef3e40e556684481b491af215b7f49b9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.105717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] RBAC: Authorizing `identity:get_user(user_id=ef3e40e556684481b491af215b7f49b9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:11.107226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:11.108865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 112/227] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:10 2022] GET /identity/v3/users/ef3e40e556684481b491af215b7f49b9 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:11.118182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:11.138173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:11.242106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8119c-bd2e-4826-825f-a5fda9a326f3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:11.270786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:11.282669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:11.283322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.283596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.287377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.287377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:11.287377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:11.290985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 116/228] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:11 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:11.297439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:11.407523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618f724-6b83-40dd-ae92-fd3f4297cfdf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:11.433764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a19a0-3fb7-4c88-ba12-7cc46b28a255 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:11.473520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:11.474251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.474544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.474825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/ef3e40e556684481b491af215b7f49b9/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.489220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] RBAC: Authorizing `identity:create_grant(project_id=c897bdb06688471f81c77286ec7fbed3, user_id=ef3e40e556684481b491af215b7f49b9, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:11.489220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:11.491685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:11.492378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.492665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.492941 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.503642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:11.509479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:11.510343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:11.511093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 113/229] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:11.518142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:11.528040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:11.545762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 117/230] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:11 2022] PUT /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/ef3e40e556684481b491af215b7f49b9/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:11.671080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bfa3a-e9da-4c38-a892-e2039df63a9a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:11.714495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:11.715132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.715465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.715751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.716480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:11.716789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:11.718243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:11.728819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 114/231] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:11.734559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:11.795278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:11.854170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bb907-313e-47ca-be75-5fc0958d1d2e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:11.895318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:11.895970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:11.896255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:11.896532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] PATH_INFO: `/v3/users/ef3e40e556684481b491af215b7f49b9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:11.897859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] RBAC: Authorizing `identity:get_user(user_id=ef3e40e556684481b491af215b7f49b9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:11.899367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:11.901519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 118/232] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:11 2022] GET /identity/v3/users/ef3e40e556684481b491af215b7f49b9 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:11.907262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:11.981457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:12.023760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3d3b6-8499-4ee4-a77e-8af97d017198 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:12.065901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f249b-4abf-4e8c-b440-d9c9652c0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:12.067213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f249b-4abf-4e8c-b440-d9c9652c0d54 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:12.067678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f249b-4abf-4e8c-b440-d9c9652c0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:12.067960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f249b-4abf-4e8c-b440-d9c9652c0d54 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:12.069164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f249b-4abf-4e8c-b440-d9c9652c0d54 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:12.070738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f249b-4abf-4e8c-b440-d9c9652c0d54 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:12.072412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 115/233] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:11 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:12.078246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:12.212057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ee10-f3b8-46b3-a42a-31cb04b8ef4b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:12.244765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:12.259217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ade84-920f-45cc-8e0a-60cb30b3318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:12.259902 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ade84-920f-45cc-8e0a-60cb30b3318f None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:12.268264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ade84-920f-45cc-8e0a-60cb30b3318f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:12.268264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ade84-920f-45cc-8e0a-60cb30b3318f None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/ef3e40e556684481b491af215b7f49b9/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:12.268264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ade84-920f-45cc-8e0a-60cb30b3318f None admin] RBAC: Authorizing `identity:create_grant(project_id=c897bdb06688471f81c77286ec7fbed3, user_id=ef3e40e556684481b491af215b7f49b9, role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:12.268264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ade84-920f-45cc-8e0a-60cb30b3318f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:12.363109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 119/234] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:12 2022] PUT /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/ef3e40e556684481b491af215b7f49b9/roles/7feebec2e0484087af48f1cb50e6f954 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:12.903607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b569be1-843a-475e-b413-04ab09359598 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:12.904116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b569be1-843a-475e-b413-04ab09359598 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:12.904222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b569be1-843a-475e-b413-04ab09359598 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:12.905873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 116/235] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:12.911500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:12.912045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:12.912045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:12.934288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:12.935467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:13.029185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 120/236] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:13.065660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b569be1-843a-475e-b413-04ab09359598 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:13.203466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b569be1-843a-475e-b413-04ab09359598 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:13.234053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:13.240304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447213b5-2e5f-44a1-bf75-dbd81eb628e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:13.240952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447213b5-2e5f-44a1-bf75-dbd81eb628e2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.241257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447213b5-2e5f-44a1-bf75-dbd81eb628e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.241693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447213b5-2e5f-44a1-bf75-dbd81eb628e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.242891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447213b5-2e5f-44a1-bf75-dbd81eb628e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:13.244570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447213b5-2e5f-44a1-bf75-dbd81eb628e2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:13.246306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 117/237] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:13.261859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:13.343965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912990eb-38e8-414e-bd46-694b54b29abe None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.344249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912990eb-38e8-414e-bd46-694b54b29abe None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.344526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912990eb-38e8-414e-bd46-694b54b29abe None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.345969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 118/238] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:13.357363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849911b-75b3-4573-949a-b03e54a11310 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.361329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849911b-75b3-4573-949a-b03e54a11310 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.361329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849911b-75b3-4573-949a-b03e54a11310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.379946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3849911b-75b3-4573-949a-b03e54a11310 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:13.380406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85dda7-2396-4a3d-8e0e-3e8a60885315 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:13.381154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3849911b-75b3-4573-949a-b03e54a11310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:13.398105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:13.420378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:13.421005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.421379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.421585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.422290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:13.423737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:13.466096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 119/239] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:13.467700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fd698-1071-4a0c-8677-fe55e49ddcc4 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.467994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fd698-1071-4a0c-8677-fe55e49ddcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.468269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fd698-1071-4a0c-8677-fe55e49ddcc4 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.473231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 120/240] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:13.478861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.479308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.479755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.499539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:13.500935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:13.502939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 121/241] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:13 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:13.509228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:13.633682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85b69b-bfd9-413c-ba0b-8c4fbc59c2cb None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:13.675345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:13.676116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.676403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.676648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.691318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:13.697510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:13.698359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:13.699104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:13.701134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 122/242] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:13.708974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:13.719578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 121/243] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:13.744923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:13.746185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:13.828138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:13.829085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4a5ff-3a05-45be-99a5-490fadfb37e0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:13.873035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:13.873707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.873984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.874257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.874981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:13.875281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:13.876677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:13.880323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e6fba-6886-433c-bd6f-fd23c862b55c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:13.888735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 123/244] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:13.895081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:13.931770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:13.935342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:13.935625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:13.935948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:13.951764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:13.959075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:13.959953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:13.960705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 122/245] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:13.968058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.017360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:14.027099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8857af-8311-4ced-99e8-1f728367bc40 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.071858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47661d-2156-4747-9590-c0351cd70301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.072778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47661d-2156-4747-9590-c0351cd70301 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.073497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47661d-2156-4747-9590-c0351cd70301 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.073774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47661d-2156-4747-9590-c0351cd70301 None admin] PATH_INFO: `/v3/users/93a509336b6c431ca0376c0bb80364dd` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.075150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47661d-2156-4747-9590-c0351cd70301 None admin] RBAC: Authorizing `identity:get_user(user_id=93a509336b6c431ca0376c0bb80364dd)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:14.077639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47661d-2156-4747-9590-c0351cd70301 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:14.079915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 124/246] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:13 2022] GET /identity/v3/users/93a509336b6c431ca0376c0bb80364dd => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:14.081151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75a87f-321d-4408-983f-d93de26918c5 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.081450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75a87f-321d-4408-983f-d93de26918c5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.081762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75a87f-321d-4408-983f-d93de26918c5 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.083640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 125/247] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:14.087283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75a87f-321d-4408-983f-d93de26918c5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.100313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739736f-7fcb-41eb-9365-e900dd08e09a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.126588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:14.144500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.145584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.146041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.147308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:14.147615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:14.149071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ee170-c23e-4ec7-8773-40a332084c53 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:14.158545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 123/248] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:14.164573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.164573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.164573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.186541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:14.187454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:14.238278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75a87f-321d-4408-983f-d93de26918c5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.276986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 124/249] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:14.277745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.278165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.279088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.279580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.280083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.281498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:14.283242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:14.286016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 126/250] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:14 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:14.304119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.362467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:14.406411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb138ce8-a507-4340-a6d3-beed3c63ce3c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.436105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:14.460324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a72b73-672a-4e2c-bc33-6bf7b6e1974a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.483702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.484361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.484644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.484919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] PATH_INFO: `/v3/users/01e0b2fd84104ef5905c8e94353ebfe5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.512582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] RBAC: Authorizing `identity:get_user(user_id=01e0b2fd84104ef5905c8e94353ebfe5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:14.514227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:14.516380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 125/251] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:14 2022] GET /identity/v3/users/01e0b2fd84104ef5905c8e94353ebfe5 => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:14.517273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.532319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.532984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.533292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.533574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/93a509336b6c431ca0376c0bb80364dd/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.537114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] RBAC: Authorizing `identity:create_grant(project_id=c897bdb06688471f81c77286ec7fbed3, user_id=93a509336b6c431ca0376c0bb80364dd, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:14.539926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:14.620832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 127/252] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:14 2022] PUT /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/93a509336b6c431ca0376c0bb80364dd/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:21:14.622125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.643764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88fe9d-0efc-4a92-a593-db84b7483c34 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.697616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.698334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.698626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.698895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.718963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:14.732495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:14.733452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:14.734214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 126/253] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:14.740122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:14.789651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:14.877887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:14.888794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada000a-9734-4e58-b45d-c5b72093d264 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.909426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f9cd7-2e5c-475e-878c-e39c3b4fc93d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:14.935800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.936445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.936700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.936972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.937731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:14.938030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:14.939423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:14.955115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 127/254] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:14.959091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe1cb-30ff-4905-9487-086a556f7fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:14.959735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe1cb-30ff-4905-9487-086a556f7fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:14.960018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe1cb-30ff-4905-9487-086a556f7fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:14.960294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe1cb-30ff-4905-9487-086a556f7fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:14.976558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:15.040802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe1cb-30ff-4905-9487-086a556f7fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:15.042316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe1cb-30ff-4905-9487-086a556f7fa0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:15.140966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:15.151065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8b4a-82a9-4808-8370-c9ea72007a5e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:15.163729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 128/255] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:15.165588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:15.167318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d92787-a7d6-4289-805f-748df823db8c None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.167812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d92787-a7d6-4289-805f-748df823db8c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.168300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d92787-a7d6-4289-805f-748df823db8c None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:15.169907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 129/256] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:15.178316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d92787-a7d6-4289-805f-748df823db8c None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:15.277914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3693e-abf0-4113-a6fb-4bd484ee35e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:15.278747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3693e-abf0-4113-a6fb-4bd484ee35e9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.279120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3693e-abf0-4113-a6fb-4bd484ee35e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.279489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3693e-abf0-4113-a6fb-4bd484ee35e9 None admin] PATH_INFO: `/v3/users/ef3e40e556684481b491af215b7f49b9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:15.286200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3693e-abf0-4113-a6fb-4bd484ee35e9 None admin] RBAC: Authorizing `identity:get_user(user_id=ef3e40e556684481b491af215b7f49b9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:15.287805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3693e-abf0-4113-a6fb-4bd484ee35e9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:15.289963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 128/257] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:14 2022] GET /identity/v3/users/ef3e40e556684481b491af215b7f49b9 => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:15.291376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.291704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.292073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:15.306957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d92787-a7d6-4289-805f-748df823db8c None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:15.324046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:15.325331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:15.382209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:15.382868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.383146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.383420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:15.384718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:15.390051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:15.392227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 130/258] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:15 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:15.392901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:15.410992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 129/259] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:15.411908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:15.484381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:15.573965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a031fc4-44f2-4da3-83a9-9066a665ec51 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:15.575076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb234-07b2-4024-bd62-eb4c0e3ff45c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:15.583634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:15.624347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:15.624980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.625290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.625570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:15.626376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] RBAC: Adding query filter params (role.id=7459c8cfcd03449b9c47588138df4243, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=01e0b2fd84104ef5905c8e94353ebfe5) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:15.626687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:15.630481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:15.641933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 130/260] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:15 2022] GET /identity/v3/role_assignments?user.id=01e0b2fd84104ef5905c8e94353ebfe5&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=7459c8cfcd03449b9c47588138df4243 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:15.643266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:15.649763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:15.650473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.650756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.651025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:15.794896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:15.796429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:15.862761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:15.864377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8e58a-a504-415b-9b69-052c5cacbeb3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:15.912653 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 131/261] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:15.918606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:15.974520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:15.975197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:15.975482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:15.975756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.013853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:16.018968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:16.020411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:16.021752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 131/262] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:16.030104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:16.033860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe8e53-9923-4ffe-8873-f81fb0e5585d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.071645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.072364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.072575 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.072856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.074106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:16.075677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:16.077799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 132/263] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:15 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:16.078358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:16.084819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:16.125651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:16.207764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea10685-2a6c-44d3-ae41-11288c886c58 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.288726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.289491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.289837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.290118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.290980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=ef3e40e556684481b491af215b7f49b9) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:16.291305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:16.292882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf57bc-cd66-4c05-9dab-f95bbfa50f61 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:16.302573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52c453-a879-4b74-bf32-4906e0d1a715 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.305509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 133/264] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:16 2022] GET /identity/v3/role_assignments?user.id=ef3e40e556684481b491af215b7f49b9&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:16.354115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.354766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.355051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.355331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.356075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:16.356387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:16.357896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:16.364110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52b91e-6779-4460-a70c-310573cad22a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.364413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52b91e-6779-4460-a70c-310573cad22a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.364815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52b91e-6779-4460-a70c-310573cad22a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.366448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 134/265] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:16.371826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 132/266] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:16.372649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.372940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.373262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.377320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:16.479019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:16.480021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:16.520093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:16.579513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 135/267] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:16.587482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf40840-73d4-4349-9e0c-5449f8916b97 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.602530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:16.630064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:16.638859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.643638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.643638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.643638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] PATH_INFO: `/v3/users/ef3e40e556684481b491af215b7f49b9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.643638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=ef3e40e556684481b491af215b7f49b9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:16.643638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:16.645626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 133/268] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:16 2022] GET /identity/v3/users/ef3e40e556684481b491af215b7f49b9 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:16.654188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:16.721287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6bd27-52a7-4c22-9d27-9a5c03ca9e8b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.759176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.759831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.760108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.760372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.773128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:16.778301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:16.779165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:16.779908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:16.782646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 136/269] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:16.787035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:16.797032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708612d-8c70-41de-a162-e6e5bcda4fb3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.834066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.834693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.834979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.835256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.855295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:16.881638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:16.912972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50040cc-1d02-46d2-b475-6f6b2926d48e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:16.921191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:16.923171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:16.961331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:16.962338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.962810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.963356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.964388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:16.964884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:16.966654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f233-4e4c-4778-b2b9-7eac37960c09 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:16.979264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 137/270] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:16.980957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cc90e-0026-4b58-b414-2853aa36fb61 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:16.981488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cc90e-0026-4b58-b414-2853aa36fb61 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:16.981956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cc90e-0026-4b58-b414-2853aa36fb61 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:16.983632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 138/271] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:16.990134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.002349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.002349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.014388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:17.015974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:17.053040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 134/272] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:17.057719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.101433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 139/273] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:17.109434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.111656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:17.196573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974a19b-1a7e-4bc4-b238-649ddabf5e0e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.200393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:17.240427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.241420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.241833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.242245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] PATH_INFO: `/v3/users/e4fd87d368fd4e01be4d46a034ba5282` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.247010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448ad85-5e16-467c-bd47-67d2be8f0d10 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.259429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] RBAC: Authorizing `identity:get_user(user_id=e4fd87d368fd4e01be4d46a034ba5282)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:17.261542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:17.263929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 135/274] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/users/e4fd87d368fd4e01be4d46a034ba5282 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:17.266008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.295336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.295984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.296262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.296692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.298639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:17.300274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:17.303046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 140/275] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:17.303886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.387887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cf2c7-89f3-4af0-82f1-3304beb4fdb7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.389608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:17.429051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.429715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf6f40-48c2-4cf7-9872-5f6202f05c2c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.430146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.430580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.431293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.433632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:17.435261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:17.437668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 136/276] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:17.438591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.473844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.474463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.474741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.475007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.475784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, user.id=e4fd87d368fd4e01be4d46a034ba5282, scope.system=all) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:17.476086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:17.477541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:17.483266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 141/277] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/role_assignments?user.id=e4fd87d368fd4e01be4d46a034ba5282&scope.system=all&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:17.490627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.495225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:17.561097 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965fd91d-33d9-4c8b-bfa6-055ea3562b3a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.614654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.615869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.616365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.617348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.618641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] RBAC: Adding query filter params (role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=ef3e40e556684481b491af215b7f49b9) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:17.619395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:17.621890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e558e0-ff77-45aa-80b7-edfe4d9309a2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:17.623300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b5ab8-d4a1-4270-9813-798cb44a347e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.637837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 137/278] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/role_assignments?user.id=ef3e40e556684481b491af215b7f49b9&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:17.637837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb56eea-9802-4bd4-bc29-3ffa2d9a844d None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.637837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb56eea-9802-4bd4-bc29-3ffa2d9a844d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.637837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb56eea-9802-4bd4-bc29-3ffa2d9a844d None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.637837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 138/279] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:17.642751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.643041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.643303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.664563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:17.666871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:17.678485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.679587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.680074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.681271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.682375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:17.684258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:17.698389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:17.750282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 139/280] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:17.775356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:17.803296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 142/281] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:17 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:17.896260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2cf2a-2b27-4856-8d98-ea8a60c42a96 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:17.936364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:17.937401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:17.937890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:17.938384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:17.951637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:17.957774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:17.958886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:17.959684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:17.962150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 140/282] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:17.969643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:18.046073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:18.098941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a04c37-1db1-40d0-9c3e-bc39fc3ee253 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:18.140045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:18.140773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:18.140951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:18.143520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:18.143520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:18.143520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:18.143857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:18.153049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 143/283] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:18.158624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:18.197733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:18.288988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8a7ba-c480-421d-bc89-695ed396e7a0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:18.324570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2c4eb-81a1-4579-91ef-92ba97961b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:18.325227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2c4eb-81a1-4579-91ef-92ba97961b44 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:18.325513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2c4eb-81a1-4579-91ef-92ba97961b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:18.325784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2c4eb-81a1-4579-91ef-92ba97961b44 None admin] PATH_INFO: `/v3/users/93a509336b6c431ca0376c0bb80364dd` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:18.326958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2c4eb-81a1-4579-91ef-92ba97961b44 None admin] RBAC: Authorizing `identity:get_user(user_id=93a509336b6c431ca0376c0bb80364dd)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:18.328418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2c4eb-81a1-4579-91ef-92ba97961b44 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:18.330400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 141/284] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:18 2022] GET /identity/v3/users/93a509336b6c431ca0376c0bb80364dd => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:18.336111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:18.454203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffe2db-e0dd-421c-a9ff-fdce0ff0a977 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:18.475924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:18.491315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e06629-7efb-4f95-b0de-add499b913ac None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:18.491603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e06629-7efb-4f95-b0de-add499b913ac None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:18.491932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e06629-7efb-4f95-b0de-add499b913ac None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:18.493362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 142/285] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:18.498404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:18.498693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:18.498975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:18.505471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:18.505471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:18.505471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:18.505471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:18.524347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:18.525570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:18.613925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:18.615437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:18.629592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 143/286] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:18.664349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:18.693228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:18.752995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 144/287] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:18.762314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:18.784052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ce190-3eb9-4af8-af6f-a0b11a5fa83a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:18.804315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:18.824156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:18.825229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:18.825763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:18.826337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:18.837607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:18.843165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:18.844047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:18.844810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:18.846943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 144/288] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:18.853562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:18.930116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c89c8f-6818-4212-b0e7-559c48fc9f7f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:18.992746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:18.998783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:18.999549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.000220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.000336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.001694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:19.003237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:19.006168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 145/289] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:18 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:19.017155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5a7c5-ad51-4ff6-b41e-277ca0fe2aaa None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:19.018246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:19.058932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:19.059829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.061044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.061496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.063101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:19.064915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:19.068076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:19.084126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 145/290] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:19.089691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:19.169996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:19.184397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf69b30-bfee-45c6-a46e-8220c1e9207f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:19.213618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734b76-e274-41f6-871b-56d290e9caab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:19.230257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:19.231198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.231672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.232151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.233157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=93a509336b6c431ca0376c0bb80364dd) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:19.233693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:19.235347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:19.248681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 146/291] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:19 2022] GET /identity/v3/role_assignments?user.id=93a509336b6c431ca0376c0bb80364dd&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:19.260735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:19.261351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.261635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.261912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] PATH_INFO: `/v3/users/01e0b2fd84104ef5905c8e94353ebfe5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.263279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] RBAC: Authorizing `identity:get_user(user_id=01e0b2fd84104ef5905c8e94353ebfe5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:19.264788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:19.267187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 146/292] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:19 2022] GET /identity/v3/users/01e0b2fd84104ef5905c8e94353ebfe5 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:19.273614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:19.426941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885c7a7-a289-42d5-b3f4-b71326b6277f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:19.466899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1c408-9f6d-4294-b5f2-bf5d9e374279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:19.467562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1c408-9f6d-4294-b5f2-bf5d9e374279 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.467848 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1c408-9f6d-4294-b5f2-bf5d9e374279 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.468125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1c408-9f6d-4294-b5f2-bf5d9e374279 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.469940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1c408-9f6d-4294-b5f2-bf5d9e374279 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:19.471432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1c408-9f6d-4294-b5f2-bf5d9e374279 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:19.474146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 147/293] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:19 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:19.480029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:19.500207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:19.595657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98297fba-85b3-41c6-a53f-ef18c86bb735 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:19.649248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8667f3-b024-43ad-ba4b-214d875b1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:19.650328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8667f3-b024-43ad-ba4b-214d875b1a6f None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.650833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8667f3-b024-43ad-ba4b-214d875b1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.651306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8667f3-b024-43ad-ba4b-214d875b1a6f None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/01e0b2fd84104ef5905c8e94353ebfe5/roles/7459c8cfcd03449b9c47588138df4243` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.655510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8667f3-b024-43ad-ba4b-214d875b1a6f None admin] RBAC: Authorizing `identity:create_grant(project_id=c897bdb06688471f81c77286ec7fbed3, user_id=01e0b2fd84104ef5905c8e94353ebfe5, role_id=7459c8cfcd03449b9c47588138df4243)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:19.656187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8667f3-b024-43ad-ba4b-214d875b1a6f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:19.752095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 147/294] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:19 2022] PUT /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3/users/01e0b2fd84104ef5905c8e94353ebfe5/roles/7459c8cfcd03449b9c47588138df4243 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:19.817928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24562dc9-f6c6-4504-a262-f1dec3d1b6c6 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.818365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24562dc9-f6c6-4504-a262-f1dec3d1b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.818786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24562dc9-f6c6-4504-a262-f1dec3d1b6c6 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.820721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 148/295] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:19.826631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:19.826972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:19.827290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:19.848277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:19.849624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:19.929265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 148/296] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:19.952550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24562dc9-f6c6-4504-a262-f1dec3d1b6c6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:19.976435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:20.059332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24562dc9-f6c6-4504-a262-f1dec3d1b6c6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:20.095565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:20.096209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.096486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.096759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.107649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:20.112601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:20.113519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:20.114288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 149/297] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:20.120048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:20.122277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:20.233745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d88c55-28a9-4eae-9572-6a60da642197 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:20.271678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:20.272311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.272597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.272876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.273637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:20.273946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:20.275416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8220250-48cb-444a-8211-acfa817675a1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:20.284667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 149/298] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:20.290576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:20.377948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:20.397907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02bd76-f7ab-4148-bca8-9b2b20eaf2c9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:20.434213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76292e8c-e301-45a7-ab90-cc885e97d172 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.434525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76292e8c-e301-45a7-ab90-cc885e97d172 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.434767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76292e8c-e301-45a7-ab90-cc885e97d172 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.436161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 150/299] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:20.441004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3cae-9cdd-4745-92dd-474a1c0ab18d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.442126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3cae-9cdd-4745-92dd-474a1c0ab18d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.442477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3cae-9cdd-4745-92dd-474a1c0ab18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.461386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b6bb3cae-9cdd-4745-92dd-474a1c0ab18d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:20.462566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b6bb3cae-9cdd-4745-92dd-474a1c0ab18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:20.484236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:20.485276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.485764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.486235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] PATH_INFO: `/v3/users/e4fd87d368fd4e01be4d46a034ba5282` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.488982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] RBAC: Authorizing `identity:get_user(user_id=e4fd87d368fd4e01be4d46a034ba5282)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:20.491417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:20.493878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 150/300] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:20 2022] GET /identity/v3/users/e4fd87d368fd4e01be4d46a034ba5282 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:20.500794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:20.534990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:20.544830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 151/301] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:20.566811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda867be-eeaf-4e4b-861d-061a842cdcde None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.567169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda867be-eeaf-4e4b-861d-061a842cdcde None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.567702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda867be-eeaf-4e4b-861d-061a842cdcde None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.569328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 152/302] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:20.572961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda867be-eeaf-4e4b-861d-061a842cdcde None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:20.621137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867ecff-41ad-4de6-867e-10d6f13cc92f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:20.657473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:20.658170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.658446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.658717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] PATH_INFO: `/v3/system/users/e4fd87d368fd4e01be4d46a034ba5282/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.660899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e4fd87d368fd4e01be4d46a034ba5282, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:20.662307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:20.682468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda867be-eeaf-4e4b-861d-061a842cdcde None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:20.717112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52d5e0-6eae-409c-8625-4e3e8c071fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:20.717773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d5e0-6eae-409c-8625-4e3e8c071fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.718059 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d5e0-6eae-409c-8625-4e3e8c071fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.718335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52d5e0-6eae-409c-8625-4e3e8c071fc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.719481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d5e0-6eae-409c-8625-4e3e8c071fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:20.720988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52d5e0-6eae-409c-8625-4e3e8c071fc3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:20.723273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 153/303] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:20.726427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.726714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.727178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.747918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:20.749088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:20.763285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 151/304] 10.222.0.66 () {58 vars in 1334 bytes} [Thu Aug 4 03:21:20 2022] PUT /identity/v3/system/users/e4fd87d368fd4e01be4d46a034ba5282/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:21:20.766390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:20.795657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:20.830227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:20.844010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 154/305] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:20.869218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:20.916646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03264e56-188b-400b-a621-69d916e105d5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:20.959509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:20.960148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:20.960431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:20.960707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] PATH_INFO: `/v3/groups` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:20.961453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:20.963729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:20.998225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec70eeb-ebf9-44e9-9280-62e0e3a1db7e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:21.035052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:21.035700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:21.035995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:21.036260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:21.057232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 152/306] 10.222.0.66 () {60 vars in 1134 bytes} [Thu Aug 4 03:21:20 2022] POST /identity/v3/groups => generated 229 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:21.058594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:21.063985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:21.064899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:21.066521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 155/307] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:21.074437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:21.157721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:21.202608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4b0b8-2622-4f03-a5a8-b5d49d51b199 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:21.251531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:21.252161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:21.252437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:21.252710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:21.253610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:21.254755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:21.255878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:21.264907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 153/308] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:21.271597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:21.333245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:21.387309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cee11-07ed-4725-8b58-efc2184f863f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:21.426013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2a30b-e790-48ca-a9dc-4f3cca958de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:21.427098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a30b-e790-48ca-a9dc-4f3cca958de9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:21.427517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a30b-e790-48ca-a9dc-4f3cca958de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:21.427929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a30b-e790-48ca-a9dc-4f3cca958de9 None admin] PATH_INFO: `/v3/users/ae1f47e8892d437a84d5a1a665ca3d3d` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:21.443164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a30b-e790-48ca-a9dc-4f3cca958de9 None admin] RBAC: Authorizing `identity:get_user(user_id=ae1f47e8892d437a84d5a1a665ca3d3d)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:21.444939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a30b-e790-48ca-a9dc-4f3cca958de9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:21.447115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 156/309] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:21 2022] GET /identity/v3/users/ae1f47e8892d437a84d5a1a665ca3d3d => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:21.453747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:21.575589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312acd25-62d3-48d0-aa80-0cb546a4c2c2 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:21.615241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:21.616151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:21.616571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:21.616982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:21.617944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] RBAC: Adding query filter params (role.id=7459c8cfcd03449b9c47588138df4243, user.id=ae1f47e8892d437a84d5a1a665ca3d3d, scope.system=all) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:21.618411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:21.620178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9facb7-2ea3-4330-91fa-fc79aef09cec None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:21.622255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:21.634104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 154/310] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Aug 4 03:21:21 2022] GET /identity/v3/role_assignments?user.id=ae1f47e8892d437a84d5a1a665ca3d3d&scope.system=all&role.id=7459c8cfcd03449b9c47588138df4243 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:21.883508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6a76a-d267-4512-ba9d-701164673987 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:21.883800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6a76a-d267-4512-ba9d-701164673987 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:21.884083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6a76a-d267-4512-ba9d-701164673987 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:21.885783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 157/311] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:21.891181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:21.891471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:21.891753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:21.914733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:21.915934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:22.008026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 155/312] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:22.031668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6a76a-d267-4512-ba9d-701164673987 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:22.156364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6a76a-d267-4512-ba9d-701164673987 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:22.173586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:22.200869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:22.202181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:22.203099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:22.203577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:22.215798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:22.222080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:22.223754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:22.224516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:22.226960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 158/313] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:22.234301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:22.354234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df174e-f517-4728-baa2-723c54e8efe2 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:22.386490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:22.394775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:22.395429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:22.395711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:22.396054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:22.396809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:22.397115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:22.398583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:22.408655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 156/314] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:22.415072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:22.527263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03d66a-75c5-4de3-8551-2ba698212997 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:22.562689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:22.563570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:22.563996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:22.564416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] PATH_INFO: `/v3/users/01e0b2fd84104ef5905c8e94353ebfe5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:22.565803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] RBAC: Authorizing `identity:get_user(user_id=01e0b2fd84104ef5905c8e94353ebfe5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:22.568520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:22.570633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 159/315] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:22 2022] GET /identity/v3/users/01e0b2fd84104ef5905c8e94353ebfe5 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:22.581240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:22.682561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32a3b0-f086-415a-a5b5-3b01db78e5c0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:22.717795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:22.718423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:22.718706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:22.718978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:22.779254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:22.789657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:22.791134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:22.897708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 157/316] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:22.903473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:22.946896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:23.011285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1fde8-ff94-4fd5-adbb-ff14f134fb56 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.046270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7af2f4-c872-4776-adc8-529385d79231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.046926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af2f4-c872-4776-adc8-529385d79231 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.047210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af2f4-c872-4776-adc8-529385d79231 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.047486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af2f4-c872-4776-adc8-529385d79231 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.048655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af2f4-c872-4776-adc8-529385d79231 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.050162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af2f4-c872-4776-adc8-529385d79231 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.052294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 160/317] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:22 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:23.058241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.058915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d792-559e-4c0f-a8c4-6bbd3af5f9ed None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.059406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d792-559e-4c0f-a8c4-6bbd3af5f9ed None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.059901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490d792-559e-4c0f-a8c4-6bbd3af5f9ed None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.061449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 161/318] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:23.067387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.067922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.068393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.086939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:23.088416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:23.159524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 162/319] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:23.175667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:23.185647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730e20e-cf04-4a33-8e54-742c045df9e9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.187296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.226756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:23.234668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.235396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.242697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.242697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.242697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] RBAC: Adding query filter params (role.id=7459c8cfcd03449b9c47588138df4243, scope.project.id=c897bdb06688471f81c77286ec7fbed3, user.id=01e0b2fd84104ef5905c8e94353ebfe5) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:23.242697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.242697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94214c01-df4b-4053-a7ec-0a9c58e31fc2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.257245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 158/320] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:23 2022] GET /identity/v3/role_assignments?user.id=01e0b2fd84104ef5905c8e94353ebfe5&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=7459c8cfcd03449b9c47588138df4243 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:23.310611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe93935-6bd2-4d9c-bbb6-d59fb230b299 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.330773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7e704-2383-4e3f-8a75-4c70c51e6577 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.331072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7e704-2383-4e3f-8a75-4c70c51e6577 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.331328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7e704-2383-4e3f-8a75-4c70c51e6577 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.333368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 159/321] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:23.338907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.339183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.339461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.352039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.353313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.353607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.353894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.358764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:23.359990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:23.368026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:23.379828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:23.380954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:23.381767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 163/322] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:23.389974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.442933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 160/323] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:23.468278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.511327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:23.521390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43f0c4-8508-4450-820d-9edfc5417035 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.562486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.563114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.563397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.563698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.564426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:23.564733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.566979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.575517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 164/324] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:23.578418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:23.582429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.594998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac422-1a64-411c-98d2-c4976bfc5374 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.634540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.635405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.635689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.635966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.637311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.638596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.640442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 161/325] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:23.646800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.696177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ad59-a42c-45ad-ac87-757ce6a0dd71 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.734284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.735088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.735376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.735940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] PATH_INFO: `/v3/users/e4fd87d368fd4e01be4d46a034ba5282` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.737125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] RBAC: Authorizing `identity:get_user(user_id=e4fd87d368fd4e01be4d46a034ba5282)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.739397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.745337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 165/326] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:23 2022] GET /identity/v3/users/e4fd87d368fd4e01be4d46a034ba5282 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:23.746752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:23.750767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf63508-f648-4048-8ed5-c222538e63f1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.776392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:23.787058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.787772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.788054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.795742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] PATH_INFO: `/v3/groups` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.795742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.795742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.848663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae882f76-91ae-47d4-a5f0-b7624f253dce None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:23.881300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:23.881916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.882197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.882472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.883253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] RBAC: Adding query filter params (role.id=e1ef11f06c1e48cba9ed25868e476dc4, user.id=e4fd87d368fd4e01be4d46a034ba5282, scope.system=all) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:23.883559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:23.884991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b055a79f-9ee7-4219-ab8b-d2bc5111b843 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:23.896681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 166/327] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Aug 4 03:21:23 2022] GET /identity/v3/role_assignments?user.id=e4fd87d368fd4e01be4d46a034ba5282&scope.system=all&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 510 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:23.897834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026ac2f-5924-49ba-b38f-e0e22b7f564b None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.898121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026ac2f-5924-49ba-b38f-e0e22b7f564b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.898395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026ac2f-5924-49ba-b38f-e0e22b7f564b None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.899607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 167/328] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:23.905191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:23.905503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:23.905781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:23.918623 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 162/329] 10.222.0.66 () {60 vars in 1134 bytes} [Thu Aug 4 03:21:23 2022] POST /identity/v3/groups => generated 226 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:21:23.927113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:23.932502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:24.011331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 168/330] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:24.033786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:24.052681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:24.147450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff67fd-d560-417e-8968-ab2670129262 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:24.187591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:24.188347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:24.188691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:24.189023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:24.200503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:24.205897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:24.207571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:24.208510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 163/331] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:24.215213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:24.235279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:24.328974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654ba60-dd65-4e07-92b2-c757ab6c7e56 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:24.367574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:24.368522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:24.368970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:24.369439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:24.370366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:24.370849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:24.372427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:24.382201 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 169/332] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:24.387690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:24.495889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:24.526390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b527bc-9cfc-4ef4-9023-6777eee97a81 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:24.568378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd1034-940a-49c1-90b4-56f9a8aa0da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:24.569030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd1034-940a-49c1-90b4-56f9a8aa0da5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:24.569337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd1034-940a-49c1-90b4-56f9a8aa0da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:24.569608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd1034-940a-49c1-90b4-56f9a8aa0da5 None admin] PATH_INFO: `/v3/users/ae1f47e8892d437a84d5a1a665ca3d3d` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:24.570782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd1034-940a-49c1-90b4-56f9a8aa0da5 None admin] RBAC: Authorizing `identity:get_user(user_id=ae1f47e8892d437a84d5a1a665ca3d3d)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:24.572230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd1034-940a-49c1-90b4-56f9a8aa0da5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:24.573877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 164/333] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:24 2022] GET /identity/v3/users/ae1f47e8892d437a84d5a1a665ca3d3d => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:24.579531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:24.667088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:24.692630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9d396-b898-40c9-b6b1-00c044794c05 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:24.728806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b367503-d141-4d28-8561-12dfec3de66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:24.729514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367503-d141-4d28-8561-12dfec3de66a None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:24.729812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367503-d141-4d28-8561-12dfec3de66a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:24.730045 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367503-d141-4d28-8561-12dfec3de66a None admin] PATH_INFO: `/v3/system/users/ae1f47e8892d437a84d5a1a665ca3d3d/roles/7459c8cfcd03449b9c47588138df4243` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:24.731520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b367503-d141-4d28-8561-12dfec3de66a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ae1f47e8892d437a84d5a1a665ca3d3d, role_id=7459c8cfcd03449b9c47588138df4243)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:24.732915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b367503-d141-4d28-8561-12dfec3de66a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:24.803516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 170/334] 10.222.0.66 () {58 vars in 1334 bytes} [Thu Aug 4 03:21:24 2022] PUT /identity/v3/system/users/ae1f47e8892d437a84d5a1a665ca3d3d/roles/7459c8cfcd03449b9c47588138df4243 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:27.014267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e859d-fc84-4f04-b509-7dcdaab91d77 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.014810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e859d-fc84-4f04-b509-7dcdaab91d77 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.015083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e859d-fc84-4f04-b509-7dcdaab91d77 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.016751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 165/335] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:27.022374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4283ebc-dc99-4896-bd70-0b78374da391 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.022814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4283ebc-dc99-4896-bd70-0b78374da391 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.023091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4283ebc-dc99-4896-bd70-0b78374da391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.031609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87596817-caa7-4170-a316-82b4aa36f0c0 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.031890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87596817-caa7-4170-a316-82b4aa36f0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.032157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87596817-caa7-4170-a316-82b4aa36f0c0 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.033534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 166/336] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:27.040433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.040713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.040989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.078416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c4283ebc-dc99-4896-bd70-0b78374da391 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:27.078625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:27.079615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:27.079720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c4283ebc-dc99-4896-bd70-0b78374da391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:27.190914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 167/337] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 03:21:27.217344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 171/338] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:27.244506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:27.258193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6669c-a1f8-43db-83e6-96fab1d339ba None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.260115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6669c-a1f8-43db-83e6-96fab1d339ba None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.260558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6669c-a1f8-43db-83e6-96fab1d339ba None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.267111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 172/339] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:27.273244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.273705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.274118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.326015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:27.327432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:27.371420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:27.407912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de0b2a-6f4e-4102-bcb5-b65a17952906 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:27.423280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:27.431053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 173/340] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:27.432099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:27.447360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa69dfb-bc10-419b-80a6-c73d604662d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:27.448452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa69dfb-bc10-419b-80a6-c73d604662d3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.448872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa69dfb-bc10-419b-80a6-c73d604662d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.449322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa69dfb-bc10-419b-80a6-c73d604662d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.462566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa69dfb-bc10-419b-80a6-c73d604662d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-6fa69dfb-bc10-419b-80a6-c73d604662d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:27.467668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:27.468704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:27.469655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:27.475224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 168/341] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:27.478941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aeb60-bd9f-46e3-ad79-a79a984973c4 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.479434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aeb60-bd9f-46e3-ad79-a79a984973c4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.479987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aeb60-bd9f-46e3-ad79-a79a984973c4 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.481768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 169/342] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:27.483261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aeb60-bd9f-46e3-ad79-a79a984973c4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:27.586747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000cdfd-a676-4abd-aed5-a9dd0ec72af4 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:27.632420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aeb60-bd9f-46e3-ad79-a79a984973c4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:27.633986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be1df6-6e9a-4f06-b18e-9e719e12f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:27.635076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be1df6-6e9a-4f06-b18e-9e719e12f11c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.636433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be1df6-6e9a-4f06-b18e-9e719e12f11c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.636900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be1df6-6e9a-4f06-b18e-9e719e12f11c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.655486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be1df6-6e9a-4f06-b18e-9e719e12f11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-42be1df6-6e9a-4f06-b18e-9e719e12f11c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:27.663858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:27.664743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:27.665556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:27.666927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 174/343] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:27.671978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.671978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.671978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.681590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:27.688003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:27.688639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.688919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.689190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.692608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:27.692608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:27.692965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:27.698934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:27.700877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:27.704105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 170/344] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:27.706116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:27.782332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:27.806535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 175/345] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:27.807455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:27.839112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb90c5-e42a-4f4e-a648-06e3f5c07bf0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:27.887002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:27.887002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.887292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.887519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.904496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:27.910564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:27.911496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:27.912526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 171/346] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:27.913178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:27.936297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64886189-bc63-4fed-9ea8-cf1237efc936 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:27.981113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:27.981913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:27.982203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:27.982696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:27.983201 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:27.983505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:27.984904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94260a-1811-4b17-acb0-a8fb8c9fcac8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:27.988514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:27.999910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 176/347] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:28.001267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b354811-f738-4902-9263-028ba2be4354 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.001562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b354811-f738-4902-9263-028ba2be4354 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.001868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b354811-f738-4902-9263-028ba2be4354 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.003066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 177/348] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:28.004098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b354811-f738-4902-9263-028ba2be4354 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.057015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bc0d9-6cae-4b40-a9bb-87cbde037ee6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.076288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.102577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.103129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.103411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.103683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.133220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b354811-f738-4902-9263-028ba2be4354 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.174987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.175719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.176001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.176987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.184470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:28.186018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:28.190908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:28.199238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:28.200117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:28.200870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:28.201479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 178/349] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:28.202058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.244754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.303141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 172/350] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:28.305238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.318588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb02aa-4def-4eda-9f1d-bf82fc829990 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.333610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.357292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.357928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.358214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.358489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.359220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:28.359527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:28.361120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18bcff-1802-4828-a38b-2f303171b9e8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:28.370088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 179/351] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:28.371313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2942-f204-435f-9ad4-cce22a7eeb5f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.371605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2942-f204-435f-9ad4-cce22a7eeb5f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.372176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2942-f204-435f-9ad4-cce22a7eeb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.393881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2c3e2942-f204-435f-9ad4-cce22a7eeb5f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:28.395710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2c3e2942-f204-435f-9ad4-cce22a7eeb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:28.432455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15c4ca-68dd-487e-b952-55e592102ea4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.470081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.470714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.471010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.471281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.482478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 180/352] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:28.483743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d830763-04f6-4f87-82e1-5f9a53d089bb None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.484034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d830763-04f6-4f87-82e1-5f9a53d089bb None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.484312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d830763-04f6-4f87-82e1-5f9a53d089bb None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.485904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 181/353] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:28.486618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d830763-04f6-4f87-82e1-5f9a53d089bb None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.502861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.560720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:28.561420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:28.609769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d830763-04f6-4f87-82e1-5f9a53d089bb None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.612076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.658658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.659293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.659724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.660001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.660847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:28.661153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:28.663384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:28.672723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 182/354] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:28.673582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.681826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 173/355] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:28.682722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.742624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.795071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ca95f-7d8c-42bc-8b21-a8ccb92a5ae7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.795935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043f624-6a54-4ce2-8847-9a0699dc7e2a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:28.835415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.836621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5eb0e-8470-4e88-96b6-d910e89758b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:28.837038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5eb0e-8470-4e88-96b6-d910e89758b3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.837297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.837417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5eb0e-8470-4e88-96b6-d910e89758b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.837590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.837680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5eb0e-8470-4e88-96b6-d910e89758b3 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.837874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.839042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5eb0e-8470-4e88-96b6-d910e89758b3 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:28.841902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5eb0e-8470-4e88-96b6-d910e89758b3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:28.842330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 183/356] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:28.843353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:28.843637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:28.843911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:28.861975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:28.864306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:28.897818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:28.916580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:28.918166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:28.944959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 184/357] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:28.945939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:28.978054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.036173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 174/358] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:29.037139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.067327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456251ee-da11-4ef8-87f4-d7dc48df8197 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.106193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.106869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.107137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.107412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.119620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:29.124552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:29.125434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:29.126237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 185/359] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:29.126870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.151792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268aae6-1fd8-4109-9bcb-3a3538753303 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.163565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.196031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.196657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.196939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.197243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.222884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.249099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d891d-b8c2-4125-98f5-834a7c555ec2 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.272251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.273772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.284560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.285214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.285513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.285788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.286940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.288394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.290070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 186/360] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.294555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.377521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 175/361] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:29.379400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.383525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.391854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da6c426-9c6b-40f1-9266-43b0309c4a61 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.426911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.427654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.427934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.428208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.431876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.434432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.435892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.437574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 187/362] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.439419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.479133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad0aa-0efb-445f-b852-754147fadb39 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.514773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.515420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.515706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.515983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.526759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:29.531634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:29.532478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:29.533259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 176/363] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:29.533887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.543532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa5b11-32de-4835-a83c-2a662c353b64 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.577973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.578681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.578971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.579250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.580420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.582045 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.583586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 188/364] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.584337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.608706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.636327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaac7a-19eb-4722-9975-b2b2a4316bf1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.649410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.670123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.670730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.671004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.671273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.671987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:29.672279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.673693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.682403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 177/365] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.683175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.686492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d98f20-6664-4483-99c1-539a43dc0e69 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.720159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.720800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.721084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.721394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.723155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.724574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.726199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 189/366] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.726952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.788600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397be9f-2117-440e-9615-3db326768b2c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.821560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.822198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.822478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.822751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.823909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.824037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.825402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.827057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 178/367] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.827808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.832304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d222f-ca27-411c-9759-bbf80d4756d4 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.866599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.867214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.867494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.867766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.868551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:29.868858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.870352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:29.873701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:29.875773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 190/368] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 260 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:29.876562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:29.955122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289492b-d30e-490c-9e25-a24e4d3d3230 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.992209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53beb9-c378-41da-b248-f94e9947a17a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:29.995193 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:29.995837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:29.996117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:29.996447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:29.997188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:29.997524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:29.998945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.014101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 179/369] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.014101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.036997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.038056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.038056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.038261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.039428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.040878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.042526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 191/370] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:29 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.043286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.070865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.138024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007400e-f5e6-4478-b324-c03bd94f68ae None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.141656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.153284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4e361-9d57-41fe-a6f6-1de3981bd334 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.174515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.176488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.176943 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.177237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.200764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.201427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.201709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.201993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.202788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=c897bdb06688471f81c77286ec7fbed3) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:30.203105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.204550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.221368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 192/371] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.221368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.275970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.277706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.342104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.356488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb94d5-28f5-4fd6-8827-44905c04100b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.396715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.397442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.397734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.398017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.399198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.400123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 180/372] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:30.400662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.401032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.402581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 193/373] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.403647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.419057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.513824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e146-2b55-4bfe-a797-cef804a51871 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.516788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88a26b-911e-4320-9918-7908f95be4ae None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.550360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.551211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.551497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.551775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.552565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:30.552877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.553287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.553932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.554216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.554421 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.554519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] PATH_INFO: `/v3/users/ae1f47e8892d437a84d5a1a665ca3d3d` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.555652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] RBAC: Authorizing `identity:get_user(user_id=ae1f47e8892d437a84d5a1a665ca3d3d)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.557083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.559029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 181/374] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/users/ae1f47e8892d437a84d5a1a665ca3d3d => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.559801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.560030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 194/375] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.560809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.597666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.680286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8fe23-8721-4751-b1ee-f6f669667a33 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.709070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.724437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.733117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.733117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.733117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.733117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=c897bdb06688471f81c77286ec7fbed3) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:30.733117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.733117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.743712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 195/376] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.744591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.778133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a428e19-25b0-4e9c-a154-0791df306895 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.840608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.841683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.842192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.842693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] PATH_INFO: `/v3/projects/94f13981caf746e383601ed3304a94e7` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.850269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] RBAC: Authorizing `identity:get_project(project_id=94f13981caf746e383601ed3304a94e7)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.852250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.854299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 182/377] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/projects/94f13981caf746e383601ed3304a94e7 => generated 344 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:30.860053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:30.882485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:30.901650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d6ec9-aab4-4298-ac27-d83331f5ba7e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:30.937095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:30.941509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:30.941509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:30.941509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:30.941509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] RBAC: Adding query filter params (role.id=7459c8cfcd03449b9c47588138df4243, user.id=ae1f47e8892d437a84d5a1a665ca3d3d, scope.system=all) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:30.941509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:30.941509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:30.957228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 196/378] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/role_assignments?user.id=ae1f47e8892d437a84d5a1a665ca3d3d&scope.system=all&role.id=7459c8cfcd03449b9c47588138df4243 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:31.029560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e07345-5de1-4c0b-b3c4-58e1a6c16955 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:31.076543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1c391-db47-4883-b04e-43ca0d6c481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:31.077639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1c391-db47-4883-b04e-43ca0d6c481b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:31.078128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1c391-db47-4883-b04e-43ca0d6c481b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:31.078602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1c391-db47-4883-b04e-43ca0d6c481b None admin] PATH_INFO: `/v3/groups/0c98d672e2dc433fad153986346cb44a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:31.086438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1c391-db47-4883-b04e-43ca0d6c481b None admin] RBAC: Authorizing `identity:get_group(group_id=0c98d672e2dc433fad153986346cb44a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:31.088292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1c391-db47-4883-b04e-43ca0d6c481b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:31.090679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 183/379] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:30 2022] GET /identity/v3/groups/0c98d672e2dc433fad153986346cb44a => generated 229 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:31.096470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:31.224018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f7990-0bf4-4700-8ccb-15b82b7326a7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:31.263760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:31.264389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:31.264674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:31.264949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:31.265775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] RBAC: Adding query filter params (group.id=0c98d672e2dc433fad153986346cb44a, role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=94f13981caf746e383601ed3304a94e7) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:31.266087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:31.267526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa12245-825b-44f9-9e9c-fd9cac275a8d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:31.274570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 197/380] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:31 2022] GET /identity/v3/role_assignments?group.id=0c98d672e2dc433fad153986346cb44a&scope.project.id=94f13981caf746e383601ed3304a94e7&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:32.255062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0309-e3de-4a3c-ba22-55aa12ea7eee None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:32.256058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0309-e3de-4a3c-ba22-55aa12ea7eee None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:32.256534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb0309-e3de-4a3c-ba22-55aa12ea7eee None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:32.258487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 184/381] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:32.265493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:32.266372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:32.266873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:32.283874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:32.294715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:32.296012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:32.372600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 198/382] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:32.412837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0309-e3de-4a3c-ba22-55aa12ea7eee None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:32.418999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:32.542121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb0309-e3de-4a3c-ba22-55aa12ea7eee None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:32.585963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e8a1f-2c61-4190-9d7a-8c39ca18a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:32.586681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8a1f-2c61-4190-9d7a-8c39ca18a754 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:32.586978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8a1f-2c61-4190-9d7a-8c39ca18a754 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:32.587265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e8a1f-2c61-4190-9d7a-8c39ca18a754 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:32.602386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e8a1f-2c61-4190-9d7a-8c39ca18a754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-1b4e8a1f-2c61-4190-9d7a-8c39ca18a754 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:32.608275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:32.609354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:32.610310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 185/383] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:32.616243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:32.748899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67152ed-45db-44ca-8032-3fa0203530c2 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:32.763909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7206-dfa5-4744-9955-23698d09c4b0 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:32.764601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7206-dfa5-4744-9955-23698d09c4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:32.765071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be7206-dfa5-4744-9955-23698d09c4b0 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:32.771225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 186/384] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:32.772985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:32.773313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:32.773598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:32.789067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:32.789727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:32.790001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:32.790267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:32.790984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:32.791283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:32.792701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:32.811579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:32.812794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:32.813568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:32.814962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 199/385] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:32.818705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:32.917742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:32.934035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 187/386] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:32.955978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.000534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a708ae-c138-492a-b3e0-34a19d7d3255 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.039554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.040215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.040508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.040790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.042289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:33.044018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:33.046051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 200/387] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:32 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:33.051394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.143300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d42df-d53c-428b-8005-4596c1e31c78 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.196327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:33.214278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.215327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.215809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.216316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.236431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:33.249055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:33.249955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:33.250731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:33.253047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 188/388] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:33.270494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cdc7c-0059-4dcb-9b7e-7ec756a23c23 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.278554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.354725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.356359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.356983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.358027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.359911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:33.362358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:33.372648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 201/389] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:33 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:33.374547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.380584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:33.463906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b9732-ee04-435f-b10a-b14fa03dc25f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.490953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cceac-0f7b-490f-be1f-7b1749251d5b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.512443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.513239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.513771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.514148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.515518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:33.515927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:33.517725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b57583-6c6f-45cb-8721-832fda642279 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:33.530552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 189/390] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:33.531016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e542f4-0cec-4edf-bb67-fb371e3f322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.531949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e542f4-0cec-4edf-bb67-fb371e3f322c None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.532077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788626-d2e0-4a61-9832-3ff03832597f None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.532455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788626-d2e0-4a61-9832-3ff03832597f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.532646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e542f4-0cec-4edf-bb67-fb371e3f322c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.532913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc788626-d2e0-4a61-9832-3ff03832597f None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.533232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e542f4-0cec-4edf-bb67-fb371e3f322c None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/groups/925adc22c0b441898c061af25bc9dbcc/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.535636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 190/391] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:33.535894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e542f4-0cec-4edf-bb67-fb371e3f322c None admin] RBAC: Authorizing `identity:create_grant(project_id=04d97b2c1dab478f84bc95fb26a2f107, group_id=925adc22c0b441898c061af25bc9dbcc, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:33.536799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95352d68-47f6-4c7d-b7a1-a17bf20d131b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.537081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95352d68-47f6-4c7d-b7a1-a17bf20d131b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.537398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95352d68-47f6-4c7d-b7a1-a17bf20d131b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.537962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e542f4-0cec-4edf-bb67-fb371e3f322c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:33.538935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 191/392] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:33.539636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95352d68-47f6-4c7d-b7a1-a17bf20d131b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.540976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:33.586102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 202/393] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Aug 4 03:21:33 2022] PUT /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/groups/925adc22c0b441898c061af25bc9dbcc/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:33.587385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e0b7-2542-4178-9462-39dcfc8e63e3 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.587675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e0b7-2542-4178-9462-39dcfc8e63e3 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.587950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8e0b7-2542-4178-9462-39dcfc8e63e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.607510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ffa8e0b7-2542-4178-9462-39dcfc8e63e3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:33.608750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ffa8e0b7-2542-4178-9462-39dcfc8e63e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:33.681954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95352d68-47f6-4c7d-b7a1-a17bf20d131b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.703114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:33.715972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 203/394] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:33.715972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.715972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.715972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.740146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.740802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.741085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.741387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.744313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:33.745823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:33.747560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 192/395] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:33 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:33.748255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.749643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:33.751185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:33.837924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 204/396] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:33.844458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.873450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c0754-8317-4c23-99d4-c086d30a4a52 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.904930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:33.912145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281813b0-f132-45c6-ac8a-f3b0efe3790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.917465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281813b0-f132-45c6-ac8a-f3b0efe3790e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.917465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281813b0-f132-45c6-ac8a-f3b0efe3790e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.917465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281813b0-f132-45c6-ac8a-f3b0efe3790e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.924501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281813b0-f132-45c6-ac8a-f3b0efe3790e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-281813b0-f132-45c6-ac8a-f3b0efe3790e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:33.930336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:33.931333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:33.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 193/397] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:33.933181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87579b64-f432-4638-a619-da4939764f2a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.933181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87579b64-f432-4638-a619-da4939764f2a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:33.933351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87579b64-f432-4638-a619-da4939764f2a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:33.937265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 194/398] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:33.937265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87579b64-f432-4638-a619-da4939764f2a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:33.959589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5659e-6a69-43c7-924e-991891310b26 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:33.989129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:33.998825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:33.999516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:33.999729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.000005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.001166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.002649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.009932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 205/399] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:33 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.009932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.056948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87579b64-f432-4638-a619-da4939764f2a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.100538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175483b-0a2c-460f-95a8-50d4bb6fdc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.101187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175483b-0a2c-460f-95a8-50d4bb6fdc93 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.101525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175483b-0a2c-460f-95a8-50d4bb6fdc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.102065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175483b-0a2c-460f-95a8-50d4bb6fdc93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.112024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175483b-0a2c-460f-95a8-50d4bb6fdc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8175483b-0a2c-460f-95a8-50d4bb6fdc93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:34.117590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:34.118445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:34.119262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 195/400] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:34.120442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.120442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.120442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.127296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7d3e7-5270-4bb2-9b61-c740562eba69 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.139832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:34.141043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:34.151921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:34.173567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.174210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.175831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.175831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.175831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:34.176050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.177294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.191270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 206/401] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.192285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.217793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 196/402] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:34.223276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.262309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:34.299929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee6bdf-32e1-457a-a9fe-24cca9a05d68 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.334910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.335106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd3cec-d904-4e5d-807d-03cc2e5a5ec4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.335678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.335878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.336156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3/groups/925adc22c0b441898c061af25bc9dbcc/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.338429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] RBAC: Authorizing `identity:create_grant(project_id=c897bdb06688471f81c77286ec7fbed3, group_id=925adc22c0b441898c061af25bc9dbcc, role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.339885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.370042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.370662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.370940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.371207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.371973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:34.372317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.373723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.382998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 197/403] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.383724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.395788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 207/404] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Aug 4 03:21:34 2022] PUT /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3/groups/925adc22c0b441898c061af25bc9dbcc/roles/7feebec2e0484087af48f1cb50e6f954 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:34.396898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.479991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:34.494866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:34.505979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcf954-b498-49ad-9060-f29f5cab6a65 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.546463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.547492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.548040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.548515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.550204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.551936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.557388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 198/405] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.557388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.560822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb1ee-e3c2-4b89-97ae-0792d884229d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.601179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.602018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.602389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.602667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.619071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:34.625242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:34.626089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:34.626862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:34.631138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 208/406] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:34.632120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.689979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c1eda-958f-41e3-890c-5a6f28dfd51a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.730726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.731316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.731597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.731869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.733033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.734536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.741052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 199/407] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.741052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.764023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac83825-4960-402d-9fa9-085a72c740dd None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.774472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:34.789629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:34.813238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.814375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.814829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.815497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.817935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.819686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.822522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 209/408] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.823493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.855693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0d1c2-3bb3-4c04-a5dd-d672b4771dd4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.896278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.896989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.897293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.897618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.898336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:34.898643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.900210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.914578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 200/409] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.915433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:34.952468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f87186-f10a-4a32-8c36-334a45ccda09 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:34.990276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42738962-c228-46f1-ad42-20238013b963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:34.991069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738962-c228-46f1-ad42-20238013b963 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:34.991362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738962-c228-46f1-ad42-20238013b963 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:34.991737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42738962-c228-46f1-ad42-20238013b963 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:34.993007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42738962-c228-46f1-ad42-20238013b963 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:34.994517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42738962-c228-46f1-ad42-20238013b963 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:34.996332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 210/410] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:34.997163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42738962-c228-46f1-ad42-20238013b963 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:35.024725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:35.032159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5333b-8ca3-4365-b5f6-bf2eb61f8a1e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:35.042567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:35.069969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:35.070934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.071359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.071778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/groups/925adc22c0b441898c061af25bc9dbcc/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.075159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] RBAC: Authorizing `identity:create_grant(project_id=04d97b2c1dab478f84bc95fb26a2f107, group_id=925adc22c0b441898c061af25bc9dbcc, role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:35.077109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:35.105246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42738962-c228-46f1-ad42-20238013b963 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:35.141519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:35.142168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.142451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.142727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] PATH_INFO: `/v3/groups/0c98d672e2dc433fad153986346cb44a` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.144566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] RBAC: Authorizing `identity:get_group(group_id=0c98d672e2dc433fad153986346cb44a)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:35.146247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:35.152343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 211/411] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:34 2022] GET /identity/v3/groups/0c98d672e2dc433fad153986346cb44a => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:35.152343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:35.169718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 201/412] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Aug 4 03:21:34 2022] PUT /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107/groups/925adc22c0b441898c061af25bc9dbcc/roles/7feebec2e0484087af48f1cb50e6f954 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:35.170892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:35.370749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de23a6-9379-4ec5-8da6-19f4bfab852e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:35.383460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:35.385475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51481ed8-af48-48ac-b523-3ae8c2b2ed5d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:35.411115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc930c4-0a81-4698-ab05-94161a03d8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:35.411853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc930c4-0a81-4698-ab05-94161a03d8ea None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.412159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc930c4-0a81-4698-ab05-94161a03d8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.412515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc930c4-0a81-4698-ab05-94161a03d8ea None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3/groups/925adc22c0b441898c061af25bc9dbcc/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.414968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc930c4-0a81-4698-ab05-94161a03d8ea None admin] RBAC: Authorizing `identity:create_grant(project_id=c897bdb06688471f81c77286ec7fbed3, group_id=925adc22c0b441898c061af25bc9dbcc, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:35.416349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc930c4-0a81-4698-ab05-94161a03d8ea None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:35.427596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:35.428439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.428439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.428707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] PATH_INFO: `/v3/projects/94f13981caf746e383601ed3304a94e7` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.431166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] RBAC: Authorizing `identity:get_project(project_id=94f13981caf746e383601ed3304a94e7)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:35.431734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:35.433670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 202/413] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:35 2022] GET /identity/v3/projects/94f13981caf746e383601ed3304a94e7 => generated 344 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:35.439416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:35.467480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:35.511915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 212/414] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Aug 4 03:21:35 2022] PUT /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3/groups/925adc22c0b441898c061af25bc9dbcc/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:35.572804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb73ddb-ef8a-4189-9eaf-4490fdea8474 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:35.623617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:35.624331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.624618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.624896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] PATH_INFO: `/v3/projects/94f13981caf746e383601ed3304a94e7/groups/0c98d672e2dc433fad153986346cb44a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.627071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] RBAC: Authorizing `identity:create_grant(project_id=94f13981caf746e383601ed3304a94e7, group_id=0c98d672e2dc433fad153986346cb44a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:35.628672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:35.702945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfae8d-8ea1-4cbd-b52f-3e08c6de7803 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.703263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfae8d-8ea1-4cbd-b52f-3e08c6de7803 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.703510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfae8d-8ea1-4cbd-b52f-3e08c6de7803 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.705220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 213/415] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:35.711044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:35.711333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:35.711613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:35.712573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 203/416] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Aug 4 03:21:35 2022] PUT /identity/v3/projects/94f13981caf746e383601ed3304a94e7/groups/0c98d672e2dc433fad153986346cb44a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:21:35.732719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:35.734053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:35.827853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 214/417] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:35.866864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:35.985136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:36.005736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b99ff-7409-496e-8c68-660ca5ce627f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:36.047323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:36.047978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:36.048261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:36.048539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:36.062316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:36.069450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:36.070322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:36.071091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 204/418] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:36.076607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:36.262512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:36.266214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0151b2-5648-4bd9-b313-8ea110d0e945 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:36.318775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:36.319410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:36.319689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:36.319966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:36.320695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:36.320997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:36.322458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb532e-fc9c-416d-a4b2-bd0782f86814 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:36.332705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 215/419] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:36.338460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:36.469052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41884d6e-a749-45a3-8068-23a8afd26ef8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:36.520536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:36.521066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:36.521371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:36.521643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:36.539937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:36.610959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:36.612865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:36.712319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea811e-f7bb-424c-b7a9-400925b99b13 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:36.712855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea811e-f7bb-424c-b7a9-400925b99b13 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:36.712855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea811e-f7bb-424c-b7a9-400925b99b13 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:36.714459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 216/420] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:36.720309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:36.720595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:36.720871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:36.744154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 205/421] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:36.745713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:36.760749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:36.762087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:36.807850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:36.885182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345930f-0e6e-41cc-bb5b-01471bf3c675 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:36.896002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 217/422] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:36.921599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:36.922306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:36.922594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:36.922875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:36.924113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:36.926232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:36.928307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 206/423] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:36 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:36.931776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:36.934287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:37.048440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15aa02-cb55-45d8-9711-e852df47be9e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.062491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:37.084072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:37.084718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.085001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.085302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.086471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:37.087917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:37.090642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 207/424] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:36 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:37.093543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:37.102362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:37.109331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46c9c4-0874-4977-b0f1-25f4c8b9f8fc None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.183058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:37.183716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.183995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.184264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.201409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:37.207472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:37.208322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:37.209083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:37.212046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 218/425] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:37.215739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:37.302144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8604794-42ca-4a0b-b6e8-6084df9fde72 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.369482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:37.372136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.373648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.374124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.375143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:37.378616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c6594-2414-4132-ae6f-3aec75a5407b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.379687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:37.381419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:37.398603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 208/426] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:37 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 611 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:37.433016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:37.434061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.434521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.434951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.435863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:37.436331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:37.438029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08cb01-1cd0-4294-8b6f-2ef1f8bf3bcc None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:37.448838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 219/427] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:37.454199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:37.527023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:37.598759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9c1b1-9b72-4762-ab25-e69cdce6dea6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.639870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:37.640834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.641391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.641881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.654751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32e66-b190-491a-a09e-c80534e9f9bb None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.655029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32e66-b190-491a-a09e-c80534e9f9bb None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.655300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32e66-b190-491a-a09e-c80534e9f9bb None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.657807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 220/428] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:37.662245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.662536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.662820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:37.692641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:37.693962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:37.705538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:37.734779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:37.736686 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:37.786106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 221/429] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:37.790277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:37.811742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:37.853924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 209/430] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:37.861464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:37.944587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90312bf5-af92-4b6c-955d-e09f5dd22f3c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.988622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977d982-3c02-4b97-9410-4a655ee7aaec None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:37.989329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:37.989985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:37.990312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:37.990590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.004505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:38.011405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:38.012247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:38.013015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 222/431] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:38.020564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.024798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.042605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d29ac8-bc94-4bc5-87a1-dd5410a41dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.043579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29ac8-bc94-4bc5-87a1-dd5410a41dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.044024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29ac8-bc94-4bc5-87a1-dd5410a41dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.044516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d29ac8-bc94-4bc5-87a1-dd5410a41dd2 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.047864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d29ac8-bc94-4bc5-87a1-dd5410a41dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.049382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d29ac8-bc94-4bc5-87a1-dd5410a41dd2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.051574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 210/432] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:37 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:38.054557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e914ec-6963-45d8-9027-2f1ce0dc2896 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.054557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e914ec-6963-45d8-9027-2f1ce0dc2896 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.054557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e914ec-6963-45d8-9027-2f1ce0dc2896 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.058643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 211/433] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:38.059438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e914ec-6963-45d8-9027-2f1ce0dc2896 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.075890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.140604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa190bd3-22c9-4387-81ba-036850909da3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.189156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.190123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.190411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.191344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.192385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:38.192718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.195088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98899a8d-3016-4ffe-a277-cf08d74a049d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.205667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 223/434] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:38.208122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.208619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.209138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.215545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e914ec-6963-45d8-9027-2f1ce0dc2896 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.229139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:38.230376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:38.252080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.252755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.253003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.254086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.255263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.256776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.263325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 212/435] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:38.263325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.298717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.317221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 224/436] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:38.317221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.359143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.385214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8f062-88bd-4b0d-98bf-ce5c535f7f5b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.412238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f94e4b-7ac5-49bf-bb51-fb09476b5262 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.421876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c39b0-63f8-4029-b771-9ee4a8e863b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.423177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c39b0-63f8-4029-b771-9ee4a8e863b1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.423460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c39b0-63f8-4029-b771-9ee4a8e863b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.423735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c39b0-63f8-4029-b771-9ee4a8e863b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.447846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.448392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.448683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.448963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.449794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=c897bdb06688471f81c77286ec7fbed3) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:38.450106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.451550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.457752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 225/437] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:38.458522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.497084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c39b0-63f8-4029-b771-9ee4a8e863b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.498619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c39b0-63f8-4029-b771-9ee4a8e863b1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.526095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.572295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de1568-0fd2-4164-ba67-427d57583b1e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.584222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.611557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 213/438] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:38.612811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922543f-7bc7-48a7-848e-e565c1d0880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.612943 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953bdb-aff4-480b-8d43-31d0bf5e16f6 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.613350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953bdb-aff4-480b-8d43-31d0bf5e16f6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.613514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922543f-7bc7-48a7-848e-e565c1d0880b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.613613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953bdb-aff4-480b-8d43-31d0bf5e16f6 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.613776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922543f-7bc7-48a7-848e-e565c1d0880b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922543f-7bc7-48a7-848e-e565c1d0880b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.614991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 214/439] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:38.618430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953bdb-aff4-480b-8d43-31d0bf5e16f6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.626125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922543f-7bc7-48a7-848e-e565c1d0880b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-7922543f-7bc7-48a7-848e-e565c1d0880b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:38.632765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:38.633693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 03:21:38.634493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:38.636439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 226/440] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:38.637775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.638056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.638330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.658645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:38.661064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:38.732297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 227/441] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:38.733044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.735354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953bdb-aff4-480b-8d43-31d0bf5e16f6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.769847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.770490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.770769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.771040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] PATH_INFO: `/v3/projects/04d97b2c1dab478f84bc95fb26a2f107` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.772202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] RBAC: Authorizing `identity:get_project(project_id=04d97b2c1dab478f84bc95fb26a2f107)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.773711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.775351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 215/442] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/projects/04d97b2c1dab478f84bc95fb26a2f107 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:38.776098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.789756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.821324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:38.840933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d2d7b-6be7-4b4c-8085-295fb2556eed None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.873215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff36bc-0048-4dca-b607-5c1e2e867313 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.875317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.875937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.876220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.876495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.877226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] RBAC: Adding query filter params (name=member) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:38.877534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:38.879304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:38.887198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 228/443] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:38.887976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.907337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:38.907981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:38.908252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:38.908548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:38.918971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:38.923822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:38.924664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:38.926056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 216/444] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:38.927217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:38.988125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1effb-c3b1-4c05-a04d-591b28dc2484 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:38.998031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.021144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.021819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.022101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.022379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.023536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.025446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.032268 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 229/445] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:39.032402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189592a-90b5-48cf-9a97-681dab3c80ba None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.032509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.039359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.065958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.066729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.067016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.067324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.126257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f96a7-0e5f-4566-8167-de9451537dfe None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.134108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.135562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.160223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.160848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.161133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.161440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.162159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:39.162467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.163853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.172240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 230/446] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:39.173376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.215076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.235627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 217/447] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:39.236458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.246114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.274070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c4c2a-0b7f-477a-9db4-b2108649cfa1 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.308916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.309569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.309857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.310136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.310928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=04d97b2c1dab478f84bc95fb26a2f107) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:39.311242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.312694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.318483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 231/448] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=04d97b2c1dab478f84bc95fb26a2f107&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:39.319262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.338461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d90fca-fa88-4579-946e-551ee4315b42 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.381656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.382610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.382887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.383269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.444325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069fc69-8009-4e1e-9e6a-839d53a981e6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.446866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.459894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.461428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.469690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.487328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50900c58-5875-47fd-8248-55874f430300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.488534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50900c58-5875-47fd-8248-55874f430300 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.488816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50900c58-5875-47fd-8248-55874f430300 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.489089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50900c58-5875-47fd-8248-55874f430300 None admin] PATH_INFO: `/v3/projects/c897bdb06688471f81c77286ec7fbed3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.491325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50900c58-5875-47fd-8248-55874f430300 None admin] RBAC: Authorizing `identity:get_project(project_id=c897bdb06688471f81c77286ec7fbed3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.492852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50900c58-5875-47fd-8248-55874f430300 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.495988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 232/449] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/projects/c897bdb06688471f81c77286ec7fbed3 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:39.500013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50900c58-5875-47fd-8248-55874f430300 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.580496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 218/450] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:39.588023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.625079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50900c58-5875-47fd-8248-55874f430300 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.665844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.666488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.666770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.667048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] PATH_INFO: `/v3/groups/925adc22c0b441898c061af25bc9dbcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.668211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] RBAC: Authorizing `identity:get_group(group_id=925adc22c0b441898c061af25bc9dbcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.669755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.671701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 233/451] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/groups/925adc22c0b441898c061af25bc9dbcc => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:39.680440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.704275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f8124-f6fd-4497-a4e0-251a5df82975 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.712287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.728558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:39.739251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.739971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.740254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.740525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] PATH_INFO: `/v3/projects/94f13981caf746e383601ed3304a94e7` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.741775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] RBAC: Authorizing `identity:get_project(project_id=94f13981caf746e383601ed3304a94e7)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.743271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.745322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 219/452] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/projects/94f13981caf746e383601ed3304a94e7 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:39.750353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:39.786951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c742c-f8df-452b-8c61-94a57fa73b87 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.819636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.820257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.820505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.820773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.821587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] RBAC: Adding query filter params (group.id=925adc22c0b441898c061af25bc9dbcc, role.id=e1ef11f06c1e48cba9ed25868e476dc4, scope.project.id=c897bdb06688471f81c77286ec7fbed3) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:39.821896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.823325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.828970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 234/453] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/role_assignments?group.id=925adc22c0b441898c061af25bc9dbcc&scope.project.id=c897bdb06688471f81c77286ec7fbed3&role.id=e1ef11f06c1e48cba9ed25868e476dc4 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:39.851534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ba0c-f53d-4373-8d6b-492696f9920a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:39.887974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60877e2f-c8f7-4834-918f-be488d7e76c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:39.888627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877e2f-c8f7-4834-918f-be488d7e76c5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:39.888906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877e2f-c8f7-4834-918f-be488d7e76c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:39.889178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877e2f-c8f7-4834-918f-be488d7e76c5 None admin] PATH_INFO: `/v3/groups/0c98d672e2dc433fad153986346cb44a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:39.890397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60877e2f-c8f7-4834-918f-be488d7e76c5 None admin] RBAC: Authorizing `identity:get_group(group_id=0c98d672e2dc433fad153986346cb44a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:39.891852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60877e2f-c8f7-4834-918f-be488d7e76c5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:39.893519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 220/454] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/groups/0c98d672e2dc433fad153986346cb44a => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:39.900449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:40.014231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797c119-bba5-4831-916f-534084ccb9cf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:40.018071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:40.058259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:40.058883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:40.059160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:40.059431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:40.060226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] RBAC: Adding query filter params (group.id=0c98d672e2dc433fad153986346cb44a, role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=94f13981caf746e383601ed3304a94e7) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:40.061298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:40.065210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69c579-fbdd-4642-b3db-90d00d07a821 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:40.077302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 235/455] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Aug 4 03:21:39 2022] GET /identity/v3/role_assignments?group.id=0c98d672e2dc433fad153986346cb44a&scope.project.id=94f13981caf746e383601ed3304a94e7&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:42.704842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b099d61-c22c-4756-8ac0-4e25aa806565 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:42.705873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b099d61-c22c-4756-8ac0-4e25aa806565 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:42.706915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b099d61-c22c-4756-8ac0-4e25aa806565 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:42.708954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 221/456] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:42.714537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b6733-ab3e-4cd8-b364-c07682795643 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:42.715385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b6733-ab3e-4cd8-b364-c07682795643 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:42.715866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b6733-ab3e-4cd8-b364-c07682795643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:42.746200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a73b6733-ab3e-4cd8-b364-c07682795643 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:42.747814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a73b6733-ab3e-4cd8-b364-c07682795643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:42.842873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 236/457] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:42.889250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b099d61-c22c-4756-8ac0-4e25aa806565 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:42.914700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8e99-c3f5-45a9-8667-b07f642b260f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:42.915338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8e99-c3f5-45a9-8667-b07f642b260f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:42.915830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc8e99-c3f5-45a9-8667-b07f642b260f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:42.917660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 237/458] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:42.922369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4895-8ab3-426a-9ccd-c20621590717 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:42.922917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4895-8ab3-426a-9ccd-c20621590717 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:42.923393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c4895-8ab3-426a-9ccd-c20621590717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:42.949597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-228c4895-8ab3-426a-9ccd-c20621590717 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:42.951150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-228c4895-8ab3-426a-9ccd-c20621590717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:42.965103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.029593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b099d61-c22c-4756-8ac0-4e25aa806565 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.041723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 238/459] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:43.043374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175c41d-7821-4fba-b041-b94dbe4eaec0 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.043906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175c41d-7821-4fba-b041-b94dbe4eaec0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.044381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175c41d-7821-4fba-b041-b94dbe4eaec0 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.046856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 239/460] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:43.052450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.052975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.053650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.074895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:43.076755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:43.087850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.088647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.089494 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.089974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.104429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.105833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.106524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:43.127469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:43.129272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:43.130110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:43.130110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:43.130110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:43.130110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:43.132445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 222/461] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:43.133891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.177224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 240/462] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:43.177224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.278659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.285617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919da70-6f4f-4ac1-9dbb-f8d525954444 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.304209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a5e81-705b-4fd4-809c-8d7ece95e97c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.321142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.321960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.322159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.322417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.334288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.335712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.340678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.341319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.341598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.341865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.342551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:43.342852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.344209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa79e7f-d0c0-4022-afe5-c456000c4bf1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:43.350681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:43.351490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:43.352306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:43.352306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:43.352306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:43.352306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:43.352306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 223/463] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:43.353512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.365662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 241/464] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:43.365662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dcf1f-8f5d-4bcd-8eaa-ca94538ecb2b None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.365662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dcf1f-8f5d-4bcd-8eaa-ca94538ecb2b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.365662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dcf1f-8f5d-4bcd-8eaa-ca94538ecb2b None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.365662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 242/465] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:43.366059 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dcf1f-8f5d-4bcd-8eaa-ca94538ecb2b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.387199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.469703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d065047-bbcd-4843-ac64-dde2e1efc1c3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.477502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dcf1f-8f5d-4bcd-8eaa-ca94538ecb2b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.505744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.506384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.506661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.506933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.511892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.512504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.512785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.513060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.513782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:43.514091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.515485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.517875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.520010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.526859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.532481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 243/466] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:43.532481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4bbbdeac-5b89-457b-a984-f850be717787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:43.541132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:43.542264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:43.543084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:43.543084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:43.543084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:43.543084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:43.543084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 224/467] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:43.543652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.543759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.543974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.563052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:43.564828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:43.626890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.634327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 225/468] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:43.635395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.669019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55f214-7d0a-4fd9-bb68-8208d911a9ad None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.712753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.713520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.713799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.714074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.714753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.716335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.739334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec84ab4-11c8-4a2e-ba80-1d959d7cd81e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:43.767430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:43.774984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:43.775596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:43.775874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:43.776143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:43.776813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:43.778368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:43.813158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 244/469] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:43 2022] POST /identity/v3/users => generated 248 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:43.813948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.877690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 226/470] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:43 2022] POST /identity/v3/users => generated 248 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:43.879546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:43.956070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab787de-f002-4c59-8a6c-6b2d9a123d36 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.004420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.005442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.006222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.006685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.008065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:44.008573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:44.010266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:44.013234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:44.032612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 245/471] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:44.039453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:44.089325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d84ac-8ecd-4129-95cb-4566734c823c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.134408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.135471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.135826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.136172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.149129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:44.150757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:44.159696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88b081-34a7-41a8-8d70-e23e95f7de5b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c6c84971-d522-445d-a626-145d9bd684a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:44.170530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:44.171388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:44.172210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:44.172210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:44.172210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:44.172210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:44.172210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 227/472] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:44.173388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f51e53-8779-4b1b-8c19-f05aec31ccd6 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.173388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f51e53-8779-4b1b-8c19-f05aec31ccd6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.173556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f51e53-8779-4b1b-8c19-f05aec31ccd6 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.175643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 228/473] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:44.184038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f51e53-8779-4b1b-8c19-f05aec31ccd6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:44.213802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958ec76-9647-4d55-85eb-8a1c4b9d1989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.214430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958ec76-9647-4d55-85eb-8a1c4b9d1989 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.214712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958ec76-9647-4d55-85eb-8a1c4b9d1989 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.214988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958ec76-9647-4d55-85eb-8a1c4b9d1989 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.215677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958ec76-9647-4d55-85eb-8a1c4b9d1989 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:44.217136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958ec76-9647-4d55-85eb-8a1c4b9d1989 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:44.233735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:44.305977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f51e53-8779-4b1b-8c19-f05aec31ccd6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.329464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 246/474] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:44 2022] POST /identity/v3/users => generated 246 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:44.330897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.331962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.332724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.345635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.346786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.347280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.347766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.348789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:44.349703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:44.352645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:44.370014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 229/475] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:44.375822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:44.379332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:44.379866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:44.479298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:44.499261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 247/476] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:44.524178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:44.560721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:44.573090 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5896f1-e648-485d-8eeb-83c39d4fd58d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.615217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.615840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.616122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.616396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.617082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:44.618568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:44.646472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfec74-e88a-4701-b907-911e2d1de68c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.683911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.684534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.684820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.685096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:44.685911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:44.687639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:44.717345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 230/477] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:44 2022] POST /identity/v3/users => generated 249 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:44.785810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.sql.core [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 03:21:44.785810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 03:21:44.785810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [parameters: {'id': '2f85695ade864ff394cc5cd9743b389b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 03:21:44.785810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80714) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application conn.query(q) Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result.read() Aug 04 03:21:44.807894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 03:21:44.808809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application next(self.gen) Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application yield resource Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application next(self.gen) Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise self.value Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application session.commit() Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 03:21:44.810325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.session.flush() Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application compat.raise_( Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 03:21:44.811174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 03:21:44.813080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application conn.query(q) Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result.read() Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 03:21:44.815014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [parameters: {'id': '2f85695ade864ff394cc5cd9743b389b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:44.817781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 03:21:44.819676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:44.826012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 248/478] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Aug 4 03:21:44 2022] POST /identity/v3/roles => generated 144 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:44.829474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:44.946918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2e0cf-0508-4334-92ab-dcf3b95db7e8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:44.986843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235df96-d34b-4f5e-a688-beffeef4734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:44.988094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235df96-d34b-4f5e-a688-beffeef4734f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:44.988607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235df96-d34b-4f5e-a688-beffeef4734f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:44.989120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235df96-d34b-4f5e-a688-beffeef4734f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:45.003377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235df96-d34b-4f5e-a688-beffeef4734f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-2235df96-d34b-4f5e-a688-beffeef4734f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:45.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:45.010208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 03:21:45.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:45.017333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 231/479] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:21:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:45.022278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:45.075599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:45.135096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bae31c-8bb7-4c80-9df4-9771dede25d6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:45.178959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:45.179857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:45.180330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:45.180794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:45.181757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:45.182269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:45.183840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609146-cacf-4804-b02f-260eeb9821c7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:45.199009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 249/480] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:46.216161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2db42-63c9-460d-8f67-377afd9e6a17 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.216658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2db42-63c9-460d-8f67-377afd9e6a17 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.216760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2db42-63c9-460d-8f67-377afd9e6a17 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.218337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 232/481] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:46.224172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.224566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.224763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.255658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:46.256856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:46.260945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093ba6-dbbb-4893-99c6-89a0adf7634d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.261279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093ba6-dbbb-4893-99c6-89a0adf7634d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.261569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7093ba6-dbbb-4893-99c6-89a0adf7634d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.263145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 233/482] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:21:46.268830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.269124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.269529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.301223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:46.301223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:46.307317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:46.374957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 250/483] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:46.396262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 234/484] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:46.403577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:46.419070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:46.478351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:46.528567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10181b20-0d4b-4f44-8e2a-28507fb717a8 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:46.566458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:46.567515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.568005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.568591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.579654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:46.585643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:46.586539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:46.587314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:46.589904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 235/485] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:46.599400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:46.606098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:46.633169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d1c69-ac21-4419-b49c-94ae7b5c3a98 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:46.707166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:46.707859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.708143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.708415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.735016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:46.742950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:46.743825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:46.744603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 251/486] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:46.755160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ada16-918a-48af-b816-9de08beef888 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:46.757071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:46.815446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:46.816574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.816855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.817126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.817875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:46.818177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:46.819561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:46.836973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 236/487] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:46.842358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:46.865056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:46.871619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f46e-badb-4996-83c0-9f725395cf98 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:46.918200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:46.922166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:46.923042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.923558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.924035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.925022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:46.925573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:46.927405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a866eb1-d574-4552-b6e2-1aae4115bff7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:46.941621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 252/488] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:46.943377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238fe4f-2f3f-4a58-8194-70eaf7044917 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.943895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238fe4f-2f3f-4a58-8194-70eaf7044917 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.944383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238fe4f-2f3f-4a58-8194-70eaf7044917 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:46.951132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 253/489] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:46.952215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238fe4f-2f3f-4a58-8194-70eaf7044917 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:46.959744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9915b-2d4c-453f-aea4-0c0c903a1cc7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:46.996423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:46.997141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:46.997565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:46.997933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.010290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.011894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-6bd09d08-e3cc-4960-8c26-1130c1e88b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:47.027182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:47.028218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:47.029238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:47.029238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:47.029238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:47.029238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:47.030620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 237/490] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:47.036289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.036673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.037050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.054909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:47.056236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:47.071509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238fe4f-2f3f-4a58-8194-70eaf7044917 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.108325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.109333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.109786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.110209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.122051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.123786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.133221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 238/491] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:47.133221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:47.161456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:47.162339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:47.163232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:47.163232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:47.163232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:47.163232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:47.165458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 254/492] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:47.166467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.191337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.201609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.267772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafee59-ba8a-43d7-a961-b2ca9c06f484 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.301064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771b91a-b4da-4447-803b-1faa02c8fc9f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.305476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.306103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.306382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.306655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.307360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:47.307786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.309174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.327174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 239/493] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:47.328062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.341716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.342703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.343234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.343721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.355487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:47.361655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:47.363252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:47.364050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:47.371256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 255/494] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:47.372469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.440461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4d4ea-d782-4463-9262-d74be1f1f873 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.448837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.453160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.478318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.479014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.479296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.479570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.480290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:47.480599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.481983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.489067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a689e33-72dd-47fc-9e68-99c67b27aac6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.500070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 240/495] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:47.500981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.529378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.530438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.530935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.531407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.540448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.542130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:47.555595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:47.556659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 256/496] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:47.557557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.617565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b1f5e-2565-4900-bd1d-8b7a200c292e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.657853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.658477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.658776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.659066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.659776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:47.660082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.661500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3bd7f-6569-4ea1-82d6-ec79879e1e8a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.668925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947599b-1692-4d9e-a558-4842e766b4a2 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.669747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 241/497] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:47.670992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6cd25-8e8a-40c8-8066-3700ecdd5e2e None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.671284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6cd25-8e8a-40c8-8066-3700ecdd5e2e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.671558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f6cd25-8e8a-40c8-8066-3700ecdd5e2e None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.677349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 242/498] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:47.677349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6cd25-8e8a-40c8-8066-3700ecdd5e2e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.706802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.710506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.714258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.715183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.715614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.716036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.724662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.726352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:47.738216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:47.740511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:21:47.741520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:47.742519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 257/499] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:47.743432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.792751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f6cd25-8e8a-40c8-8066-3700ecdd5e2e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.829309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.829928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.830208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.830482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.831886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:47.832199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.833654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210ddd7-2483-443d-8789-5ba40aadfffd None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.853343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75659b6c-0473-417f-9a01-a1400df3fc3a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:47.870856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 243/500] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:47.872062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ef24-ab39-4691-867e-7d1350623d57 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.872347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ef24-ab39-4691-867e-7d1350623d57 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.872618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1ef24-ab39-4691-867e-7d1350623d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.888811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:47.889488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.889781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.890055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.890951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-11b1ef24-ab39-4691-867e-7d1350623d57 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:47.892156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-11b1ef24-ab39-4691-867e-7d1350623d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:47.900702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:47.902209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:47.917706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:47.918532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:47.919347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:47.919347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:47.919347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:47.919347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:47.919347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 258/501] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:47.919606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.960783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 244/502] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:47.962083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1d5fc-ca78-4f05-9d9d-2942cda1fd4b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:47.962371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1d5fc-ca78-4f05-9d9d-2942cda1fd4b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:47.962650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1d5fc-ca78-4f05-9d9d-2942cda1fd4b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:47.963898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 245/503] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:47.964589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1d5fc-ca78-4f05-9d9d-2942cda1fd4b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:47.974591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:47.984183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.033437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e01fe-7da2-4c17-88d9-ed56546042af None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.066859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.067402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.067683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.067958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.068447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1d5fc-ca78-4f05-9d9d-2942cda1fd4b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.068749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:48.069055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.070552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.102224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.102827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.103109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.103377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.107184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 259/504] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:48.108003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.176741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.178256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.195514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.209769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.212644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b5037-1f87-4167-8a78-aae20aa07ab7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.247177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.247796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.248081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.248356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.249050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:48.249388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.250796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccff76e-95f2-4cf1-b46d-459161467827 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.262604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 260/505] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:48.263776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.264067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.264345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.281395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:48.282593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:48.282802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 246/506] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:48.283637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.352151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 261/507] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:48.352905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.382887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fdd2e-1f1c-48d1-8eb7-cd8526e1e080 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.400130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.418458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.419133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.419461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.419732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.430671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:48.436021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:48.437092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:48.438065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:48.441521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 247/508] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:48.442200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.447855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.457862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed653a-86d2-4821-aad7-905b4872edd8 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.494768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.495383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.495663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.495936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.541726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd87f00-4cfe-4e4d-867c-6dee0635f74c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.563035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.564519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.574800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.575423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.575708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.575966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.583092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.584526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:48.597722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:48.598641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:48.599524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 248/509] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:48.600059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.637320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.653972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.664424 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 262/510] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:48.665466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.701820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3765aa7-c990-430a-9f1a-8a5c2146b051 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.735084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.735722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.736002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.736276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.748084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.749580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:48.764120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:48.765092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:48.766050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:48.766050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:48.766050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:48.766050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:48.766252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245e79f-0255-4d78-87a7-fca5064fc953 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.766322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 249/511] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:48.766322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.798911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.799549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.800552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.800889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.811296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.812892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:48.827095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:48.828096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:48.829127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:48.829127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:48.829127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:48.829127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:48.829127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 263/512] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:48.829127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.856013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.868457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577e33b-c801-4874-92b4-25602f15dace None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.873434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:48.902928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.903466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.903737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.904013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.904717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:48.905076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.906485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.915533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 250/513] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:48.916312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:48.932794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670d234-7296-45f5-aa30-0802c83c7df1 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:48.966520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:48.967136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:48.967419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:48.967695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:48.968519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:48.968824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:48.970325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:48.988611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 264/514] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:48.993397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.017941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c00aa-1902-4bb5-bfcd-a1b9a0d713b8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.051113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.051737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.052015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.052285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.062467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.063907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:49.077984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:49.078833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:49.079665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:49.079665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:49.079665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:49.079665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:49.079665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 251/515] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:49.079916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.083209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.089567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.092489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad5f28-33f8-4834-8172-10312526d6ee None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.126123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.126755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.127014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.127285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.128150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:49.128454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.130120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.141429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 265/516] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:49.142225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.189591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a88664-c97f-4043-8956-7b1833686ee1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.222574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.223382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.223541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.223808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.224492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:49.224789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.226201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.238064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 252/517] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:49.238853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.241830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ccbe4-9a8a-407e-bb3d-b42f4a4aa4e5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.277738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.278436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.278734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.278992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.289813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.291286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.300359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:49.305958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:49.322051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:49.322864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:49.322864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:49.322864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:49.322864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:49.322864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 266/518] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:49.322864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.323326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.340234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d832c-7329-4159-b23d-7e5d8570cfc5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.374426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.375040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.375320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.375595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.409322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cec9d-2dfd-4817-9036-2c3d89c29b16 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.443029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.443862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.444407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.444514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.444747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.445021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.445746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:49.446052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.447763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.463052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 267/519] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:49.463872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.513390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.535492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.554048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 253/520] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:49.555046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.563966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c7188-50c6-41a1-af85-8140715205a5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.599191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.599823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.600100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.600371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.606885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.608324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:49.620826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:49.621699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:49.622558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 268/521] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:49.623008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.657401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a930-7598-4fb2-a15c-bbe0649c6bae None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.691663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.692274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.692562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.692836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.693667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.695047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.722902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094a504-62a9-47ca-88fb-2b6a47aa2860 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.750933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.756263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.756866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.757145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.757447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.758132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:49.758438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.759820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.771802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 269/522] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:49.772568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.779274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 254/523] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:21:49 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:21:49.782889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.802395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:49.895755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f5262-2de1-4ea9-871b-f9cb02c54407 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.900024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61337957-b32f-4917-b38e-973c8a790f0a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:49.935920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.936561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.936838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.937105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.938619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:49.939259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:49.939533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:49.939800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:49.944775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.946200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.951411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:49.952859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:49.958555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:49.959348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 270/524] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:49.960862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:49.969634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:49.970480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:49.971326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:49.971326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:49.971326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:49.971326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:49.971326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 255/525] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:49.972131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.029876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.048335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.078653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad51722-01a5-4ae4-b517-f9404d75e9f6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.081367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451444ee-98ed-48e9-9929-e6426c37dec8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.116429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.117046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.117392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.117622 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.118367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.118665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.119279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.119948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.120164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.120256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.120488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.129548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.130994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.143022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 271/526] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.143022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:50.144107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:50.144950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 03:21:50.146533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:50.149386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 256/527] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:50.150230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.256214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bca76e-7301-4632-be08-8a7964028cd6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.260617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3221ff-f394-471e-b49e-78667892b9f7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.272289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.292078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.293897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.294512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.294793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.295111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.295805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.296113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.297671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.298469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.299155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.299360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.299660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.301036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.301725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.302597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.311369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 272/528] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.312081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.324175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 257/529] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.324175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.420559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e40a-456d-483e-86c2-ffdcceb8c920 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.428355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451061ff-4d7a-4a77-a7e3-aaa95741789f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.456608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.457241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.457523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.457804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.458615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=2e4a8d38d08c4619812d98a544510b29) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.458900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.460339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.463423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.464039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.464318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.464597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.465415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.465724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.477674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 273/530] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/role_assignments?user.id=2e4a8d38d08c4619812d98a544510b29&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.477877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.477968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.498511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 258/531] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.499413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.526514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.556772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.622058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de63a8-0b58-420f-833a-3c43494af902 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.640034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f16f36-2a40-4949-bb7e-91ca4d3de40b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.667339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.668018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.668300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.668572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.669462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=428e53d5cb9e4a2996295b94e364ece4) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.669775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.671251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.677275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 274/532] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/role_assignments?user.id=428e53d5cb9e4a2996295b94e364ece4&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.677886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.686513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.687512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.688012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.688489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.698443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:50.712185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:50.713079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:50.714244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:50.716457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 259/533] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:50.726272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:50.882353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8043-ec18-46df-9681-2ad3da03967f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.895281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.907772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:50.913781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20aabd9-8085-46d7-9b5e-712500ac90b5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:50.928389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.929146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.929460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.929736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.930723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:50.931033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:50.933219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:50.953770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:50.954413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:50.954696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:50.954971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:50.958961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 260/534] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:50.965887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:51.063027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:51.064715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:51.148429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e105398-e056-49e3-b411-318c0a693b2a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:51.217916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:51.218905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:51.219397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:51.221117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:51.222322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=09fbb54e522c47d89de166682742336f) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:51.223016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:51.225351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:51.236701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 261/535] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/role_assignments?user.id=09fbb54e522c47d89de166682742336f&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:51.257289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:51.262240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 275/536] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:51.268105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:51.306063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:51.402108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f028e-5c56-4416-a68d-87dfaef58427 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:51.444861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:51.445558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:51.445837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:51.446108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:51.469768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:51.471272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:51.487541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:51.488503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:51.489347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:51.489347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:51.489347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:51.489347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:51.489347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 262/537] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:51.494426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:51.633328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53c1d-059c-4d4f-ae3a-8d177f08d466 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:51.678349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:51.679314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:51.679787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:51.680252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:51.681136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:51.681664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:51.683235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:51.701812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 276/538] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:51.707778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:51.842816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:51.865896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6242f4-27eb-4c3e-aa84-b937c4652d38 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:51.918067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:51.918715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:51.918997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:51.919270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:51.932236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:51.933665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-0671712c-15d8-4d84-aa76-53515c5680e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:51.954743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:51.955615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:51.956473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:51.958963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 263/539] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:51.964188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:52.038746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a0b7-60f5-4b82-a2f0-44aab91c685a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.039384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a0b7-60f5-4b82-a2f0-44aab91c685a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.039866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a0b7-60f5-4b82-a2f0-44aab91c685a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.041471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 264/540] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:52.047381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.047666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.047940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.057224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:52.075148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:52.076375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:52.105548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c11a65e-f558-4c0a-956e-513f1cd89107 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:52.161727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:52.162362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.162654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.165693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.165938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:52.166255 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:52.171227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:52.183403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 265/541] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:52.189689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 277/542] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:52.198252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:52.220012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:52.341724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7e5f1-828f-429e-9f2a-29ab176e091b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:52.366875 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:52.386375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:52.391581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:52.392441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.392905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.393487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.414465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:52.417452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320e9f8-3310-412c-9a66-cf118a1dd9b3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:52.429251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:52.430149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:52.430913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:52.433646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 278/543] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:52.439354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:52.463252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:52.463884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.464166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.464439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.465261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=8c54c5182ce245e49cdbf25f1254238e) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:52.465576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:52.467025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:52.480562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 266/544] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:52 2022] GET /identity/v3/role_assignments?user.id=8c54c5182ce245e49cdbf25f1254238e&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:52.558589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f158b84-261e-48b0-b3a3-9a49f1059722 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:52.597438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:52.598436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.598912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.599392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.600332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:52.600827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:52.602435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:52.611904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 279/545] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:21:52.618272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:52.762030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3f57d-9f32-4596-8494-2fd31363b308 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:52.799248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:52.799978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:52.800263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:52.800542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:52.807502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:52.812213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:52.813713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:52.828878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:52.835320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:52.836142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:52.836142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:52.836142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:52.836142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:52.836142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 267/546] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:52.837118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:52.891489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:52.963709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85313052-38f0-4e36-817a-ed6bad9d57f3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.008355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.009133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.009443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.009720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.010423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:53.010731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:53.012178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba895b-dc49-4ab3-b32d-1bedf285216e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:53.030510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 280/547] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:53.036790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.067180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0530a-d024-4b3c-97c8-3bab4fa08cd8 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.067498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0530a-d024-4b3c-97c8-3bab4fa08cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.067807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0530a-d024-4b3c-97c8-3bab4fa08cd8 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.069160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 281/548] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:53.086626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.086916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.087197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.110665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:53.112224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:53.203525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 282/549] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:53.243780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f28dd-6ae8-4f06-b5e2-ea2cbd54219d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.258729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.306128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.306998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.307437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.307851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.324145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:53.330527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:53.333071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c9de1278-50b1-4985-9e58-5314dc532f7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:53.355133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:53.356249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:21:53.357116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:53.359650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 268/550] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:53.361108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ca70a-3b95-483c-92f7-884f83d32fd6 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.361636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ca70a-3b95-483c-92f7-884f83d32fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.369069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ca70a-3b95-483c-92f7-884f83d32fd6 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.369069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 269/551] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:53.369069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ca70a-3b95-483c-92f7-884f83d32fd6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.422770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:53.423375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ca01b-84df-4132-90c3-0a1bb429dd83 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.462222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f552050-363f-4ca4-8b75-55e8b5604669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.463055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f552050-363f-4ca4-8b75-55e8b5604669 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.463408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f552050-363f-4ca4-8b75-55e8b5604669 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.463755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f552050-363f-4ca4-8b75-55e8b5604669 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.475759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f552050-363f-4ca4-8b75-55e8b5604669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:53.479348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ca70a-3b95-483c-92f7-884f83d32fd6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8f552050-363f-4ca4-8b75-55e8b5604669 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:53.489593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:53.490642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:53.491430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 283/552] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:53.492832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.493342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.493827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.514205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:53.515881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:53.521673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.522298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.522579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.522853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.523655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:53.523964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:53.525435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:53.544916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 270/553] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:53.545783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.606049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 284/554] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:53.606866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.645124 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:53.679711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c812bc7-e403-4b5d-aac5-83a04eb1edc9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.689146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:53.715184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.715786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.716067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.716338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.717053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:53.717389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:53.718773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:53.728882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 271/555] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:53.730283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.738938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c6689-c9ec-4805-befd-17f0fb9e80a7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.778462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.779084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.779362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.779633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.841575 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337ddca-5b9e-4af5-9848-bf7d2e468597 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:53.854320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:53.855866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:53.879783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3c859-bfd3-4214-8f09-b0f5d7ff71f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:53.880422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3c859-bfd3-4214-8f09-b0f5d7ff71f2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.880549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:53.880715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3c859-bfd3-4214-8f09-b0f5d7ff71f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.880970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3c859-bfd3-4214-8f09-b0f5d7ff71f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.891691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3c859-bfd3-4214-8f09-b0f5d7ff71f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-98a3c859-bfd3-4214-8f09-b0f5d7ff71f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:53.898200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:53.901371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:53.902275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:53.904332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 272/556] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:53.905452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34af9e0-9838-424c-904e-0c4c264fac86 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:53.905736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34af9e0-9838-424c-904e-0c4c264fac86 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:53.906011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34af9e0-9838-424c-904e-0c4c264fac86 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:53.907332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 273/557] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:53.908008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34af9e0-9838-424c-904e-0c4c264fac86 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:53.941756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:53.966488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 285/558] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:21:53.971746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.019161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34af9e0-9838-424c-904e-0c4c264fac86 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.053805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.054437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.054717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.055047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.066566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.068042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.074663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b715bf-a284-47a0-a05a-09a5f83ea113 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-3298e5ed-4a60-415d-beb9-7ea73b117d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:54.083277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:54.084229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:54.085116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:54.085116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:54.085116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:54.085116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:54.088880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 274/559] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:54.090072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.090357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.090630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.100990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.109648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:54.110843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:54.113488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.114105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.114383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.114650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.115366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:54.115666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.117030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.125969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 286/560] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:54.132326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.197234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 275/561] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:54.197234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.205588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.262816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e30d8f-a492-4b70-9dc1-c3e27084332c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.296634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.297420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.297700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.297970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.303629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d504d9-75a3-4374-bbf4-34ea068f9c8c None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.309318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.310772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:54.325738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:54.326746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:54.327589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:54.327589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:54.327589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:54.327589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:54.327589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 287/562] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:54.333264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.346129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.347366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.347653 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.347915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.348611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:54.348914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.350334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.362738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 276/563] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:54.366150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.373691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.442291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a90e5-fa45-4e53-94f5-b0188fd5ba8d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.478504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.479145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.479780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.479780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.484227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.490548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.492026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:54.507564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:54.508496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:54.509453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:54.509453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:54.509453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:54.509453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:54.509453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 288/564] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:54.509716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.520147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85f766-c22d-4329-9123-2b1edd6ba1b1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.554297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.554934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.555215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.555484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.567763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:54.574815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:54.575722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 277/565] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:54.582122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.617831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.629981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc65f6-077f-41ad-bf41-48e055defaaf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.666256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.666872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.667154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.667448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.668116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:54.668419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.669837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.681990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 289/566] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:54.682792 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.693529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a392b-3f61-494d-8424-530e935bfa0b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.723930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.728725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.729395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.729684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.729952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.736952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.738784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:54.750522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:54.751525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 278/567] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:54.752440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.790098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5cba5-a143-4cfc-a108-13ca59d876cb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.823406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.824015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.824301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.824574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.825291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:54.825597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.826968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.838781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 290/568] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:54.839751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.851754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b794e-8a55-4c8a-afbe-f10c8b316693 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.853013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.886024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.886635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.886913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.887183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.887893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:54.888195 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.889579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9aae7-8d66-4140-814c-92d1641192db None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:54.900387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 279/569] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:54.901616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5792-ed9e-4ac6-843b-90fb57e1104b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.901906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5792-ed9e-4ac6-843b-90fb57e1104b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.902183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5792-ed9e-4ac6-843b-90fb57e1104b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.903587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 280/570] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:54.904100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5792-ed9e-4ac6-843b-90fb57e1104b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:54.948345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd80831-a762-454a-ac46-ea1432a9123f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:54.960747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:54.981917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:54.982554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:54.982836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:54.983151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:54.990210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:54.991661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:55.003302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:55.004635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 291/571] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:55.005691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.010128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5792-ed9e-4ac6-843b-90fb57e1104b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.043106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.043718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.043998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.044270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.045062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:55.045395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.046823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.064436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 281/572] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Aug 4 03:21:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:55.065836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.077772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.110378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc04fd-13b8-478c-bad0-ff6a6088628e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.145533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.146252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.146537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.146811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.154092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.155543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.162158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eae978-3423-4d23-bace-64dc8ca8d244 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-ee5b657a-c55c-437a-85a4-b1c2364b888c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:55.166601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:55.167497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:55.168418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 292/573] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:55.169889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.169889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.169889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.176401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.187790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:55.189193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:55.197741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.198422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.198654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.198927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.209330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.210793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:55.226039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:55.227067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:55.228122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:55.228122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:55.228122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:55.228122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:55.231825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 282/574] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:55.232681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.260461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 293/575] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:55.261402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.319822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.333723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3ebb3-a8c0-462d-b249-11036b2c76bf None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.359948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bad60-f2bd-41d5-93e3-d8815d0af62d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.366283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.366898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.367176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.367445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.368197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:55.368499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.369966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.382677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 283/576] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:55.383489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.399795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.400439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.400723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.400999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.411275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.412745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:55.427115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:55.428180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:55.429271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:55.429271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:55.429271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:55.429271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:55.429271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 294/577] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:55.429271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.457672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.529810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfa3f8-8bde-418a-91f3-35c50aef8c46 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.531208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052b412-d57e-46b9-90a9-333e0548b527 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.540236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.563261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.563908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.564187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.564459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.565686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:55.566214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.566331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.566816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.567093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.567358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.567459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.568145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:55.568444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.569882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.579041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 295/578] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:55.579836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.591779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 284/579] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:55.591779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.671270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.681649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c3340-26be-4c29-bd3c-5c843c961917 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.689552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c20b82-d809-4005-b516-a4d7339c45c7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.715680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.716249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.716524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.716796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.724629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.725263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.725606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.725887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.726674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=d647c7fcd5fb490db5302a9ead7ab330) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:55.726983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.727529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:55.728427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:55.732467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:55.733401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:55.734196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 296/580] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:55.734946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 285/581] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/role_assignments?user.id=d647c7fcd5fb490db5302a9ead7ab330&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:55.734946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.738238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.777984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:55.849645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdc4ae-1329-423d-935e-d6e8398bc77c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.855094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d0a0-04ea-414d-9fff-995b77786bc5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:55.886459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.887422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.887702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.887972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.894671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:55.895525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:55.895633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:55.895839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:55.896542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:55.896849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.897417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:55.898284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.898853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:55.911409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 297/582] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:55.912072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:55.920465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:55.921433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:21:55.922352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:55.928136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 286/583] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:55.928136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:55.933926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.009285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.021243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33986629-beb0-4a62-88eb-ffeca03b7409 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.032664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb71d4-8a9d-409a-a46a-14ea8b1d2791 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.058229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.058881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.059165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.059443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.068565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.069619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.070173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.070461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.070826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.071186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:56.071492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.072299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.072889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.086936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 287/584] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:56.086936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:56.087156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:56.087963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:56.088814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:56.088814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:56.088814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:56.088814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:56.088814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 298/585] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:56.088814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.160379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.204249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3da09-d859-4d49-b661-77a47d3d0a1e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.210318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363367bb-f82b-4e50-a3b3-fb33b57ac613 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.240619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.241897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.242306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.242593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.246384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.247008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.247291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.247570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.248367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:56.248676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.250185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.250739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.252326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:56.265522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:56.266547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:56.268255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:56.269262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.269341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 288/586] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:56.269341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.275555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 299/587] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:56.277410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.374593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893dc240-2851-43a2-9b9a-0eb2425197d4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.385323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49889408-4f17-4df5-962b-288efc5ca934 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.403805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.413556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.414207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.414491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.414767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.425957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.426583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.426929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.427210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.427306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.427905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:56.428211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.428701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.429642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.443129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 300/588] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:56.443952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:56.445043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:56.445929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:56.446857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:56.446857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:56.446857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:56.446857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:56.448369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 289/589] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:56.449306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.514607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.558234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a4f38-46bb-4ec8-a4ce-2cbd4d4635df None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.560829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c53fab-4d7e-4a56-b7c0-b7e797cb0d82 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.592551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.593231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.593518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.593789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.595713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.596716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.597232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.597716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.598672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:56.599202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.600841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.604487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.605960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.616336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 301/590] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:56.617440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:56.621113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:56.622132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:56.623031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:56.623031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:56.623031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:56.623031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:56.623031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 290/591] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:56.623562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.658795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.729133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87493f43-18e0-4ce7-bf01-5551afb23815 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.729757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc20f-a22d-463a-b0cf-503dc962d917 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.750544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.765758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88867885-b572-4202-be28-7302a355624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.766386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88867885-b572-4202-be28-7302a355624d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.766667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88867885-b572-4202-be28-7302a355624d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.766767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.766955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88867885-b572-4202-be28-7302a355624d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.767312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.767594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.767868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.768624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:56.769088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.770553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.773930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88867885-b572-4202-be28-7302a355624d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.775369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88867885-b572-4202-be28-7302a355624d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.782788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 291/592] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:56.783580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-88867885-b572-4202-be28-7302a355624d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:56.790281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:56.791158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:56.791997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:56.793693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 302/593] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:56.794602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88867885-b572-4202-be28-7302a355624d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:56.887524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:56.896029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bffe0-498e-43fa-a353-59a8f3e3c9fa None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.906570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88867885-b572-4202-be28-7302a355624d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:56.931441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.932102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.932381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.932659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/428e53d5cb9e4a2996295b94e364ece4/roles/b94a012ed1954740858f831257bb8811` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.943985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:56.944605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:56.944895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:56.945163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:56.945892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:56.946196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.947253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=428e53d5cb9e4a2996295b94e364ece4, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:56.947581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.948679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:56.959374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 303/594] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:56.959982 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.001798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.024685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 292/595] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:56 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/428e53d5cb9e4a2996295b94e364ece4/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:57.025648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.078790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f204c0-f069-4b44-b440-6a713547abfb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.119831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.120626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.120988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.121362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.130849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.132596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:57.146574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:57.147795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:57.148714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 304/596] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:57.149324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.165654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b05222-d5ac-45f4-87d8-2101f165edf2 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.225191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.226142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.226612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.227135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.228058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:57.228525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.230033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.243171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 293/597] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:57.243822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.246564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.270522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.273714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b8989-e1a1-40b6-a8e5-768f9b061466 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.319563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.320249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.320721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.321014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.329961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.332169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:57.347619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:57.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 305/598] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:57.350180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.359054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457d511-fb9a-42a1-a9ae-cd6954fe47e7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.401706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.402567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.403040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.403452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.404501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:57.405014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.406777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.431796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 294/599] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Aug 4 03:21:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:57.433060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.472004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8aa6-6c63-48ab-b66f-fb8aea6529de None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.509781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.510841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.511321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.511790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/2e4a8d38d08c4619812d98a544510b29/roles/b94a012ed1954740858f831257bb8811` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.519643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.531292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=2e4a8d38d08c4619812d98a544510b29, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.532764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.560998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56358b60-8288-4da5-9e73-e49112835f12 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.597140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 306/600] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:57 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/2e4a8d38d08c4619812d98a544510b29/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:57.604472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.604671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.604671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.604671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.604671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.605026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:57.605026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.605026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.622404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 295/601] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:57.630427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.638770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.796355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acde430-7890-41ef-9de2-b7bfd45071da None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.819312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3076f-df39-4c12-9b2a-c06800344c8b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:57.835608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.836266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.836548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.836829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.848263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.849700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.855031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-296d0fc6-3ade-4218-82e1-71dbcc8d035b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:57.866854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:57.871678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 307/602] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7a8b-b969-4063-8483-e2bb907736dd None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7a8b-b969-4063-8483-e2bb907736dd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.872580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a7a8b-b969-4063-8483-e2bb907736dd None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.874384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 308/603] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:57.878803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb99158-e342-4531-8964-29b93bd25702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:57.878803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb99158-e342-4531-8964-29b93bd25702 None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.878803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb99158-e342-4531-8964-29b93bd25702 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.878803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb99158-e342-4531-8964-29b93bd25702 None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/09fbb54e522c47d89de166682742336f/roles/b94a012ed1954740858f831257bb8811` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:57.880260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a7a8b-b969-4063-8483-e2bb907736dd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:57.899762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb99158-e342-4531-8964-29b93bd25702 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=09fbb54e522c47d89de166682742336f, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:57.901677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb99158-e342-4531-8964-29b93bd25702 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:57.951083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:57.996405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 296/604] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:57 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/09fbb54e522c47d89de166682742336f/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:21:57.997750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:57.998138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:57.998536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.000625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a7a8b-b969-4063-8483-e2bb907736dd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.020403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:58.021765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:58.057177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.058415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.059317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.060083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.061090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:58.061628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:58.063464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:58.083970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 309/605] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:58.092890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:58.115560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 297/606] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:21:58.134026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:58.263230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:58.266381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cb088-1f83-4623-99ef-a4eb69fef2ee None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.288396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645327a-6253-4187-8fd4-860943ec1514 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.312625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.312818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.312989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.313311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.317554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:58.328585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.329276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.329564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.329847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.342796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:58.345405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:58.346110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:58.351399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:58.352342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:58.353161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 298/607] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:58.366459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:21:58.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:58.377435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:58.379873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 310/608] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:21:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:58.386654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:58.576917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32077521-90cf-49b6-93cd-1806c43eb65e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.586017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bc390-1bbb-42cc-b5b9-0184fb94d8ff None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.632529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.638373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.638373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.638373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.638373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:58.638373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:58.638373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:58.655184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 311/609] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:21:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:58.659184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.659953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.660326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.660695 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.661867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:58.662752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:58.671729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:58.673330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:58.689600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 299/610] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:58.695448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:58.733568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:58.742324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:58.834117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecb694-3cc5-4e8e-ae50-58e3de9ae730 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.855519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401531d-b22c-4225-a8ef-a8cb26cb38dd None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:58.901164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.901852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.902129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.902398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.914528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:58.915215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:58.915488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:58.915757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/8c54c5182ce245e49cdbf25f1254238e/roles/b94a012ed1954740858f831257bb8811` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:58.927621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:58.929100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:58.946433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=8c54c5182ce245e49cdbf25f1254238e, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:58.947883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:21:58.956512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:58.957794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:58.958914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:21:58.958914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:21:58.958914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:21:58.958914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:21:58.958914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 300/611] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:58.964896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:59.029351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 312/612] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:21:58 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/8c54c5182ce245e49cdbf25f1254238e/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:21:59.075497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:59.121704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626d347-ac1f-41cf-b5af-5eaa210dc33f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:59.188407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:59.189044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.189349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.189624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:59.190910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:59.191325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:59.192732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:59.212690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 301/613] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:59.218847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:59.341957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:59.351523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631080-9b49-45f0-8bd1-e5abe1afe088 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:59.387599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:59.388175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.388457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.388731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:59.397758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:59.399594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-6da97d65-a53a-4155-847b-a5be000a4999 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:21:59.413184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:59.414407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:59.415671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 313/614] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:21:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:21:59.422602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:59.461226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec5ca-c3c6-41ad-9302-8534e78257ed None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.461226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec5ca-c3c6-41ad-9302-8534e78257ed None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.461226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec5ca-c3c6-41ad-9302-8534e78257ed None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:59.463243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 314/615] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:21:59.469611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.473221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.473221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:59.491682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:21:59.493146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:21:59.578188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 315/616] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:21:59.604023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:59.647370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1204e-9276-488e-9992-e298f1f27479 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:59.705083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:59.717480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca97cb8-e3f8-4e20-96c3-ccd079cfc9ad None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:59.758038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:59.758437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:59.758956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.759073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.759321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.759489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.759591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:59.759936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:21:59.760382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:21:59.760680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:21:59.762154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:21:59.765854 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:21:59.772078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:21:59.779001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:21:59.780399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:21:59.781254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:21:59.783119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 316/617] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:21:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:21:59.790054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:59.793695 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 302/618] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Aug 4 03:21:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:21:59.799170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:21:59.959882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f8728-7ca4-4985-83c3-91ceb3554649 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:21:59.998111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:21:59.998918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:21:59.999283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:21:59.999632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.000985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:00.001402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.003910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.013169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 317/619] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:21:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:00.019718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.024706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.033106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dde30-1e14-4f22-b023-ad096553954c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.079118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.080380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.080770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.081631 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.096345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.098660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-dc0b70ee-19da-46fa-b202-cccc3d4cfa49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:00.119694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:00.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:00.121814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:00.121814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:00.121814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:00.121814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:00.124185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 303/620] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:00.124185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451f44-19c2-4172-adeb-f3080fe9a05a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.124364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451f44-19c2-4172-adeb-f3080fe9a05a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.124576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19451f44-19c2-4172-adeb-f3080fe9a05a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.126142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 304/621] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:00.134302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19451f44-19c2-4172-adeb-f3080fe9a05a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.146375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d49b3e-084a-421b-b033-529b2916be98 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.189735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.191041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.191331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.191610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.197338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.206393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.208121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-352c793b-fc92-44e6-bb37-23eaffb8009d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:00.228535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:00.229720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:00.230616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:00.230616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:00.230616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:00.230616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:00.233267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 318/622] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:00.236788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.238035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.239666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.261999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:00.263610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:00.272080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19451f44-19c2-4172-adeb-f3080fe9a05a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.309061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.310951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.311435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.311876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.312787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:00.313328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.314942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.331320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 305/623] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:00.332912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.337386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.351812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 319/624] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:00.354750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.457705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c6752-bdd9-428f-9e2b-f4ec2eef3512 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.458795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc948cc3-6acc-4c21-a9d3-9a76a6e4c777 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.462293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.494748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.495313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.495595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.495871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.496468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.497191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.497496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.497882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.498694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:00.498996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.500464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4da71-6116-4c05-9685-10d60d442f56 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.503919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.505400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.521647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 306/625] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:00.521647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229286-7015-44b8-a848-b9ebf4834821 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.521647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229286-7015-44b8-a848-b9ebf4834821 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.521647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc229286-7015-44b8-a848-b9ebf4834821 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.521647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 307/626] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:00.522141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:00.523125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:00.524128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:00.524656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229286-7015-44b8-a848-b9ebf4834821 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.524737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 320/627] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:00.524737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.571917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.627074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2323cd-4479-4416-86ec-4678e811f2e1 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.634698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc229286-7015-44b8-a848-b9ebf4834821 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.663200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.663849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.664132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.664408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.671520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.672971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.674044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b6261-c911-40a6-a73c-587b125e769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.675399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b6261-c911-40a6-a73c-587b125e769a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.675810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b6261-c911-40a6-a73c-587b125e769a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.676461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b6261-c911-40a6-a73c-587b125e769a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:00.685222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:00.686123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:00.687063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 321/628] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:00.689244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b6261-c911-40a6-a73c-587b125e769a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:00.690762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-fe0b6261-c911-40a6-a73c-587b125e769a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:00.695530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:00.696606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:00.697607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:00.701302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 308/629] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:00.702716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.702999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.703274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.722416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:00.724222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:00.743180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.805307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 309/630] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:00.805307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.819654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632d630-45bd-4222-be18-2d025f539af3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.827312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:00.856228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.856937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.857316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.857663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.858489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:00.858876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.860416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.873446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 322/631] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:00.874354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.914180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8aa05-fd77-43fe-9957-ae1848b5e27e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.949991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:00.950609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:00.950890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:00.951163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:00.952077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:00.952384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:00.953988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:00.978429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 310/632] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:00.978429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:00.981968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c43f-126d-4cc9-9df7-77927c853daa None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:00.992953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.019178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.019893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.020253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.020586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.021416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:01.021785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.023231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.036886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 323/633] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:01.037841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.068103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.082071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c583-3d18-431d-b359-05f9b4bb38b0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.117300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.117950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.118298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.118632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.130092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:01.134980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:01.136059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 311/634] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:01.137077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.152926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fd738-18bb-4720-9e78-25662e30126d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.193486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.194064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.194344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.194618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/d647c7fcd5fb490db5302a9ead7ab330/roles/b94a012ed1954740858f831257bb8811` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.210466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=d647c7fcd5fb490db5302a9ead7ab330, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.211913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.220073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.247300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee352c85-0394-4404-a97a-4b8c1ac152e8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.273233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 324/635] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:01 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/d647c7fcd5fb490db5302a9ead7ab330/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:22:01.273233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.297541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7deb02-9c7a-42b2-a7a6-ec962f4eb79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.299018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7deb02-9c7a-42b2-a7a6-ec962f4eb79e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.299376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7deb02-9c7a-42b2-a7a6-ec962f4eb79e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.300005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7deb02-9c7a-42b2-a7a6-ec962f4eb79e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.372736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.401985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec9917-8d09-4958-b12d-e8a9ab8ec8ab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.418739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7deb02-9c7a-42b2-a7a6-ec962f4eb79e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.420580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7deb02-9c7a-42b2-a7a6-ec962f4eb79e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.447839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.448504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.448767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.449041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.460509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.462018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:01.477223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:01.478118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:01.478979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:01.478979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:01.478979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:01.478979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:01.483180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 325/636] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:01.484026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.496822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.536619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 312/637] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:01.536619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc8707-19fd-4d5b-95f7-40ba63503637 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.536619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc8707-19fd-4d5b-95f7-40ba63503637 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.536619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc8707-19fd-4d5b-95f7-40ba63503637 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.536619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 313/638] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:01.537110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc8707-19fd-4d5b-95f7-40ba63503637 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.594374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348147f5-ff1c-49b7-8c67-041b68c74c66 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.627464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.634111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.634740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.635466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.635757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.636679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:01.636983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.638406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60dd6b-a9bc-4f31-a3ab-aeabaf771c87 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.652437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 326/639] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:01.652613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc8707-19fd-4d5b-95f7-40ba63503637 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.652707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.652707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.652707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.669768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:01.670967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:01.689522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.690315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.690597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.690895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.691548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:01.691846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.693236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.710622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 314/640] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:01.710622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.749377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 327/641] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:01.749377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.750709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.823983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1676857-0729-4e75-b4ed-2c49a8493d0f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.859978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.860617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.860897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.861170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.861488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1add35-bae6-4fa2-b34e-8c100c42fab7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:01.872662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.874153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.886230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:01.889500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:01.890392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:01.891282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:01.891282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:01.891282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:01.891282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:01.891282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 315/642] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:01.891562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.903046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:01.903716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:01.903996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:01.904272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:01.915470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:01.917303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:01.932672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:01.933861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:01.934770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:01.934770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:01.934770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:01.934770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:01.934770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 328/643] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:01.935055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:01.985915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.002133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61deb5-cd88-47d7-8c68-515879384672 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.040204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95130b7b-1b9c-4560-bc4f-92e071be12da None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.042258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.043233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.043233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.043419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.051425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.052865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:02.064672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:02.066194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 316/644] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:02.067205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.082646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.083288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.083673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.083953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.094148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:02.099709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:02.100649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 329/645] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:02.101876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.141809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.183468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4457a4-efe4-460c-9433-e787d2b8c6cc None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.217546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d205ab-6118-4eb8-9d4a-11a1c9592b00 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.219473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.220376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.220503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.220738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.221518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:02.221801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.223275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.236704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 317/646] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:02.238606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.245877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.257468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.258200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.258484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.258757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.259452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:02.259750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.261120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.277787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 330/647] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:02.277787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.357028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac656a6-6d2c-4540-a25a-8c637d518c89 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.373539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.381618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cda853-848f-40c2-99ac-5c8dc5aca5e6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.394904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.394904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.394904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.394904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.395634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:02.395942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.397482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.416230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 318/648] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:02.417057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.424274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.425664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.425974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.426244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.426963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:02.427260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.428628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.443426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 331/649] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:02.444093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.483692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.523481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ed511-7041-4380-a556-9d8aa3da33c4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.551837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fef72c-1eec-4f75-bbc6-5f08a2635c38 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.557328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.557970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.558246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.558515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.567353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.568922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:02.581440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:02.582629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 319/650] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:02.583499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.588701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.589380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.589664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.589939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.598481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.600060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:02.612081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:02.612986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:02.613939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 332/651] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:02.615363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.633056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.693397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56e515-d197-44ba-95b5-0419b92c8e94 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.715919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.719787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd7cb9-c842-46b6-9b98-8da693daf612 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.728483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.729122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.729415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.729687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.756217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.757252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.757713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.758180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.769189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.770883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:02.785771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:02.786704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:02.787565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:02.787565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:02.787565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:02.787565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:02.789192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 333/652] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:02.790125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.805871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.807319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.864370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.902190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a4a3-709a-48b5-aa3f-a64d6d4373d4 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:02.919552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 320/653] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:02.919552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:02.941694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:02.944952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:02.945555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:02.945832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:02.946100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:02.946871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:02.947132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:02.948566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:02.960645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 334/654] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:02.961541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.025471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd193a-4261-429f-8a09-6dd19111d69b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.062895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.063598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.063872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.064165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.064923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:03.065881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.067514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.073398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491402a-3a65-4308-9a17-8d0a9452a82a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.085912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 321/655] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:03.090887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.105447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.113807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.114434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.114709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.115005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.117528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:03.117528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.117528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.130151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 335/656] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:03.130970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.189984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.209624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d2bc8-da49-4733-9a2b-37ce23b69234 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.241986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3cbcb-8532-4606-a156-27c20d633558 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.246284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.246925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.247258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.247533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.258637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.260220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:03.276312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:03.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:03.278103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:03.278103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:03.278103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:03.278103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:03.278103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 322/657] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:03.278367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.283062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.283717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.284631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.284922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.285764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=428e53d5cb9e4a2996295b94e364ece4) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:03.286077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.287531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.294006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 336/658] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/role_assignments?user.id=428e53d5cb9e4a2996295b94e364ece4&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:03.294659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.360244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.403147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ba2c5-6d61-49ff-b7b5-f3ebff4e1ad6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.423508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fec0a-57e5-4c93-90f2-7b0887ed9f61 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.437767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.444241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff98c481-5e60-422a-a091-b7fbce3b09f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.444728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98c481-5e60-422a-a091-b7fbce3b09f1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.445013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98c481-5e60-422a-a091-b7fbce3b09f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.445319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff98c481-5e60-422a-a091-b7fbce3b09f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.463788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.464361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.464645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.464929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.474553 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.476287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:03.488752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:03.489781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 03:22:03.490799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:03.492872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 337/659] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:03.493856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.522454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98c481-5e60-422a-a091-b7fbce3b09f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.523919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff98c481-5e60-422a-a091-b7fbce3b09f1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.629651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6462-6fe1-419b-8638-299faebe07df None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.637088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 323/660] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:03.637088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fec9b6-3cc4-4330-973d-5e470f3278bf None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.637088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fec9b6-3cc4-4330-973d-5e470f3278bf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.637088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fec9b6-3cc4-4330-973d-5e470f3278bf None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.637088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 324/661] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:03.637088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fec9b6-3cc4-4330-973d-5e470f3278bf None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.648632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.669430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.673237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.674017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.675886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.676524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.677571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:03.678077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.679860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.699662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 338/662] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:03.700435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.745075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fec9b6-3cc4-4330-973d-5e470f3278bf None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.781075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.781718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.781998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.782264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.783089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:03.783389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.785514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7919002-babb-424a-924f-f9f0ef924afb None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.804230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 325/663] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:03.805861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.806352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.806834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.812748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c084-ad5a-47e1-94d1-90daefb580ab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:03.825588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:03.826770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:03.851951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:03.852650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:03.852934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:03.853249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:03.864159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:03.866610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:03.884703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:03.885605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:03.886468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:03.886468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:03.886468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:03.886468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:03.887105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 339/664] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:03.887750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.904830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 326/665] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:03.906009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:03.918279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:03.938097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.000752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a96a-11f2-4edb-b182-b3bdcc7330d0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.018767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494e975-2409-4ca7-94a7-32b48b720ca0 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.039495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.040140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.040422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.041225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.050871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.052328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.055754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.056570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.057022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.057495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:04.064020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:04.064944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 340/666] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:04.067330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.135774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.137932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.156856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.167407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.176731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab03df8-02ee-49ab-ae96-fc7a68d15a0f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.214207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.214854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.215133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.215405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.216083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:04.216384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.217818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.231351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 341/667] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:04.232145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.248967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 327/668] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:04.250066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.342685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85f61d-b05a-4ec1-ad4b-e98ab4996d43 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.355413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29796e9-5498-4b39-9f1f-82fc1c94d7a9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.375204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.387062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.395949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.395949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.395949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.396211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.396211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.396211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.396211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.396511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:04.396511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.396511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.401178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:04.411048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 328/669] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:04.411048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:04.411301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:04.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 342/670] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:04.414638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.421489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.514181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10a6fd-c6ae-4014-a244-c56328e2f2bc None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.525942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf728bb-5734-429e-a743-deb41164ced8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.552284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.552930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.553250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.553516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.560360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.561849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.562054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.562685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.562968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.563243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:04.573985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:04.574831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:04.581308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:04.581735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.581808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 329/671] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:04.581873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.581939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:04.597809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:04.598651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:04.599515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:04.599515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:04.599515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:04.599515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:04.599515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 343/672] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:04.599777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.630350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.661683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.692268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e2869-0e09-4959-bbf9-40dc035a4949 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.706673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c1512-1efd-47b2-9067-37379719c15c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.726931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.727544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.727824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.728096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.728927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=2e4a8d38d08c4619812d98a544510b29) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:04.729263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.730879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.745229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 330/673] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/role_assignments?user.id=2e4a8d38d08c4619812d98a544510b29&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:04.745229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.751106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.751891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.752251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.752712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.754053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:04.754529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.755984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.766325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 344/674] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:04.767222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.867415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5428743-f066-4480-b5bf-e12dea3e9153 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.870743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.907013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.907765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.908137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.908479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.909528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:04.909876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.910852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff666de-0505-47fa-8bb9-8d90e940ecb6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:04.911580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.925924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 331/675] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:04.926872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:04.937005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:04.959942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:04.960829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:04.961289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:04.961887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:04.962756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:04.963225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:04.964759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:04.980596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 345/676] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:04.981773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.054580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70a30f-0d09-4e38-9c5f-8469163f9188 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.098189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.099048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.099398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.099739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.109409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcbfc5d-5344-413e-ad41-5a955f242a78 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.112599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.114359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:05.131148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:05.132026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:05.132890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:05.132890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:05.132890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:05.132890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:05.135082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 332/677] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:05.136098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.157594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:05.159867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.163376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.163376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.163376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.163376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=09fbb54e522c47d89de166682742336f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:05.163376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.163877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.171679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 346/678] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:04 2022] GET /identity/v3/role_assignments?user.id=09fbb54e522c47d89de166682742336f&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:05.172626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.233946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:05.301920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3bdbd-2389-4ec9-b226-36e93be05277 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.337442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f17064-b96b-46b1-b7c1-912a89e3c31a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.344199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.344902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.345186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.345525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.347462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:05.347799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.349226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.369031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 347/679] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:05.370527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.399254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.399827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.400104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.400371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.410661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.412401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c88db313-8944-47b7-9adc-e40b34d78334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:05.425496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:05.426430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:05.427301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 333/680] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:05.433429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99020ea-0039-482a-a19a-968acd2c9525 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.433712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99020ea-0039-482a-a19a-968acd2c9525 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.433997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99020ea-0039-482a-a19a-968acd2c9525 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.435544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 334/681] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:05.436246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99020ea-0039-482a-a19a-968acd2c9525 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.499351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7eaddd-fab1-4da2-a1e6-a1fbcfe4437b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.501097 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:05.526619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:05.546779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.547703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.547981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.548253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.558402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.559908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.563511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99020ea-0039-482a-a19a-968acd2c9525 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-68105a5a-e01a-42cd-9bbe-e1f02694fcc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:05.572359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:05.573617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:05.574842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 348/682] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:05.576867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.576867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.576867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.602075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:05.603263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:05.615422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.616427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.616988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.617294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.618066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:05.618370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.620716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.639924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 335/683] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:05.640795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.683785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 349/684] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:05.684704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.776570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2436c-3287-48d1-85c8-6f6cd53dd56b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.813813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.814630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.815772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.816141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.817052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:05.817472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.819005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.822314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:05.823788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:05.837652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bce8c8-f200-480e-aeab-ffb434fb035e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:05.838979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 336/685] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:05.840073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.883860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:05.884812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:05.885095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:05.885402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:05.886610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=8c54c5182ce245e49cdbf25f1254238e) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:05.886925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:05.888675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:05.901247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 350/686] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/role_assignments?user.id=8c54c5182ce245e49cdbf25f1254238e&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:05.901247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:05.986498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709eb1a2-62bc-4000-9309-78fdbb911e9c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.025699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.026566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.026846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.027118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.027764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.029267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:06.042148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:06.043138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:06.043945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 337/687] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:06.050183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:06.085090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4b4a1-0e73-4717-9717-76f2aa2d8283 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.133247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.133878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.134158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.134432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.144186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:06.192119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:06.199430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02446b41-65dc-446a-bfde-b66ba9ee5746 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.240528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.241517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.241995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.242461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.243362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:06.243868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.245557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.252107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 338/688] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Aug 4 03:22:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:06.256731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.256731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.257655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:06.365337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 351/689] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:06.367066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:06.424427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:06.461619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e88ff-8be7-49de-96f3-98b49ea7b2e3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.468719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021d4bb5-2386-4e37-b620-8c2ced582cb0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.505636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:06.506617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.507272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.507723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.508007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.519300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.520762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.530299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.531231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.531695 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.532182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.533058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:06.533565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.536280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:06.545234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:06.546109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:06.546970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:06.546970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:06.546970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:06.546970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:06.546970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 352/690] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:06.546970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:06.559238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 339/691] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Aug 4 03:22:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:06.661461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d0b4-8254-4db3-8e0a-f118e7e20a03 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.672023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:06.709518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.710775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.711347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.711960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.713214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:06.713521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.714922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.732951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 353/692] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:06.738894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:06.872205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe2621-7f21-4042-986a-c9e67fb6836b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:06.910138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:06.910785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:06.911063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:06.911331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:06.919983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:06.921487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:06.934109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:06.935029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:06.935883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 340/693] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:06.946121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:07.126640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e828750-3e85-4b63-a6fa-1dbae7e52fff None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:07.213942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:07.214892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.214892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.215185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.215872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:07.216179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:07.217663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4d381f-2571-4998-b59e-da78b1e74ee9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:07.234877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 354/694] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:07.245060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:07.290881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057648c-a0dc-4c84-bc14-ecccf98d86ee None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.291277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057648c-a0dc-4c84-bc14-ecccf98d86ee None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.291426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057648c-a0dc-4c84-bc14-ecccf98d86ee None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.293243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 355/695] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:07.298338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.298637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.298904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.329138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:07.331326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:07.342068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:07.381620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:07.429582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eb2b4-5f7f-43cb-b004-5850aee6b135 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:07.465435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 356/696] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:07.493888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:07.494683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.495063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.495432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.496327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=d647c7fcd5fb490db5302a9ead7ab330) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:07.496732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:07.498290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:07.505586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 341/697] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:07 2022] GET /identity/v3/role_assignments?user.id=d647c7fcd5fb490db5302a9ead7ab330&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:07.515400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:07.660168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe56e8-5d25-4091-bdc3-3939362cfe22 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:07.715835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:07.716655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.717032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.717422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.733520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:07.735169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b994ab95-76fc-41de-beec-c519d82f6cb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:07.757866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:07.758966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:07.760018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:07.760018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:07.760018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:07.760018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:07.761266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 357/698] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:07.767113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:07.808470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409d099-4ac5-4647-a888-2c277d4d27e8 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.808800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409d099-4ac5-4647-a888-2c277d4d27e8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.809022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409d099-4ac5-4647-a888-2c277d4d27e8 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.810852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 358/699] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:07.823044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.823335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.823691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.844690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:07.845917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:07.849656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:07.879669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ce63d-7fa9-4d77-9fb3-bc0b09e7d81a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:07.891283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:07.917584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:07.918127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:07.918407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:07.918677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:07.919379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:07.919683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:07.921255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:07.932843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 359/700] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:22:07.941308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 342/701] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:07.943149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:07.958440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:08.072483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e287d-013b-492f-83db-8bdaf264dca4 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:08.082625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd2e39-38fb-421d-b6eb-5583b0707d0a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:08.111065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:08.111715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:08.112824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:08.113603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:08.114308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:08.116320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:08.123482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:08.124127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:08.124453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:08.124731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:08.132009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:08.139079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:08.144784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:08.145751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:08.146563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:08.147716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 343/702] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:08.151375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:08.198504 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:08.215085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 360/703] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:22:07 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:22:08.301089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0888b-3d8f-4e8c-aa7e-5dd2711bdc36 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:08.356809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:08.357882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:08.358361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:08.358857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:08.359783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:08.360282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:08.362452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:08.373472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 344/704] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:08.381477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:08.493545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43282828-1b5f-4ad8-8128-9360c2ced799 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:08.533165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:08.534841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:08.535320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:08.535808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:08.547049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:08.548807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:08.564770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:08.566893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:08.567772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:08.567772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:08.567772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:08.567772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:08.569968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 361/705] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:08.578321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:08.685913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:08.696247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08ef3-f4a7-46c5-88b7-11a6f5671f86 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:08.732450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:08.736503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:08.736889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:08.737261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:08.738060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:08.738440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:08.739901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:08.753039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 345/706] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:08.758734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:08.827226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:08.899191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b5f2e-de53-45a9-99ea-218afe71abc7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:08.941982 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:08.942683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:08.942966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:08.943236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:08.954234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:08.955794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-2aca92f5-ff45-4e5a-aa44-c2cc1264b649 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:08.970648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:08.972983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:08.974284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 362/707] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:08.982281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.032836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523883-af7e-4f8c-8321-c10c4bb43f35 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.033226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523883-af7e-4f8c-8321-c10c4bb43f35 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.033508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6523883-af7e-4f8c-8321-c10c4bb43f35 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.036435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 363/708] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:09.041540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.041821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.042091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.079652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:09.080894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:09.145019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889dd337-d0fa-4064-a3b9-75efbebeddcc None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:09.181947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 364/709] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:09.197296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:09.198127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.198560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.199042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.200000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:09.200462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:09.202447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:09.223300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.224932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 346/710] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:09.230194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.248651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:09.307493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:09.389882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96a402-d318-47e9-934b-b066c9174dba None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:09.429004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a56a58-8496-4a00-8050-55129e4ae7e2 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:09.441909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89e25d-6b48-4db5-a66b-5d52b1ecc4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:09.442681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89e25d-6b48-4db5-a66b-5d52b1ecc4ee None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.443058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89e25d-6b48-4db5-a66b-5d52b1ecc4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.443425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89e25d-6b48-4db5-a66b-5d52b1ecc4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.474827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:09.475481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.475844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.476041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] PATH_INFO: `/v3/services/network` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.476681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:09.478384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-c8ca11e9-af1d-4810-99e8-0b86d4c8a31a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:09.489949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:09.491112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:09.492085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 365/711] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:09 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:09.492893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba146ed-4d72-4d62-a2ea-5eaece6405d7 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.492893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba146ed-4d72-4d62-a2ea-5eaece6405d7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.493234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba146ed-4d72-4d62-a2ea-5eaece6405d7 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.498563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 366/712] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:09.498563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba146ed-4d72-4d62-a2ea-5eaece6405d7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.531858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89e25d-6b48-4db5-a66b-5d52b1ecc4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:09.533713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89e25d-6b48-4db5-a66b-5d52b1ecc4ee None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:09.545041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:09.619881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba146ed-4d72-4d62-a2ea-5eaece6405d7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:09.623457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:09.641139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 347/713] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:09.641139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.641139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.641139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.659850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:09.661132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:09.671229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:09.672108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.672580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.673093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.674076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] RBAC: Adding query filter params (name=network) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:09.675247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:09.677279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:09.682841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 367/714] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:09.684096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.741289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 348/715] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:09.742197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.795281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd13cd-2122-47a5-80bc-55e60362223f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:09.817601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:09.834428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:09.835240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.835620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.835994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.848330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:09.850006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:09.864682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e68a5-d482-42e1-9d7c-e5b8e89c2de4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:09.865634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:09.866735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:09.867731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:09.867731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:09.867731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:09.867731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:09.873024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 368/716] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:09.873927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.894835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:09.903535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:09.912947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:09.912947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:09.912947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:09.912947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] RBAC: Adding query filter params (type=network) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:09.912947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:09.912947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:09.914119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 349/717] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:09.915121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:09.986661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8701f2b-dc9c-403c-8b74-a3223306a7c3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.031209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaef9fa-f290-4c09-af35-8304e2e8247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.031953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaef9fa-f290-4c09-af35-8304e2e8247b None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.032315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaef9fa-f290-4c09-af35-8304e2e8247b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.032880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaef9fa-f290-4c09-af35-8304e2e8247b None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.035316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaef9fa-f290-4c09-af35-8304e2e8247b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:10.035650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaef9fa-f290-4c09-af35-8304e2e8247b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:10.049760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42054908-289d-4c21-b022-f5a3587ffc50 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.057571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:10.087682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.088322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.088602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.088918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.089678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:10.090025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:10.091421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:10.109761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 350/718] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:10.122414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:10.148942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 369/719] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:09 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:10.239521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b759cb-9570-40a6-b395-d29d1d342136 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.249438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c08e-bbea-4d9c-9f51-df5d5e5f7c1a None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.250080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c08e-bbea-4d9c-9f51-df5d5e5f7c1a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.250080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0c08e-bbea-4d9c-9f51-df5d5e5f7c1a None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.251622 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 370/720] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:10.259160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.259478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.259706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.279833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:10.281142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:10.328396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.329068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.329388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.329725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.346478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:10.348147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:10.357327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 371/721] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:10.368528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:10.370768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:10.372891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 351/722] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:10.375721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:10.392169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:10.396307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:10.437543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:10.508910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda3d0e-720e-4772-96ad-97d896acfd5b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.531752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc7c8c-a713-481e-af25-a9c745608c5b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.553393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.554320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.554801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.555281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.556209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:10.556684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:10.558784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:10.574890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.575249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 372/723] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:10.575634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.575982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.576266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.580616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:10.587682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:10.592696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:10.593661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:10.594469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:10.599345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 352/724] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:10.600585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:10.680536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:10.693925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:10.717413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b525c-1077-48d7-8efb-a1d8a352c18c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.725340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50366c32-21ef-485c-abc5-994b21230d4c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:10.764304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.764937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.765226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.765516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.766253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:10.766557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:10.768496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:10.786598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 353/725] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:10.792305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:10.798157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:10.798783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.799061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.799945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.800736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=09fbb54e522c47d89de166682742336f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:10.804063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:10.805568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd9813-4b8a-4f7d-9a12-eedc5854c8c2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:10.821717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 373/726] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:10 2022] GET /identity/v3/role_assignments?user.id=09fbb54e522c47d89de166682742336f&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:10.889336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fc975-b4ec-48d1-82fe-b71e5f5e4cf9 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.889679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fc975-b4ec-48d1-82fe-b71e5f5e4cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.889899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fc975-b4ec-48d1-82fe-b71e5f5e4cf9 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.891290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 374/727] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:10.897043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:10.897365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:10.897634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:10.934457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:10.935719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:10.965254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b54e96-e0ad-4ba7-8b24-6b7aad0e087d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.035159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.036229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.036720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.037188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.055601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:11.056193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 375/728] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 03:22:11.057249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:11.059361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:11.075165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:11.077277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:11.078122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:11.078122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:11.078122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:11.078122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:11.078122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 354/729] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:11.082131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.083978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.161739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:11.206854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19612d4-abba-47ef-8abc-8dec9531d60b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.249505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.251128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.251567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.251993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.252971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:11.253711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:11.255398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:11.269027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 376/730] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:11.274749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.350170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aae43-d69a-4561-b5ca-091a7106c2be None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.394134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d73e9-fac5-4941-a287-f668bbc39620 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.420724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:11.425583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.426217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.426497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.426771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.442268 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.442918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.443192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.443461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.446055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:11.454149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:11.455072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:11.456660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 355/731] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:11.460797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:11.460797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:11.461939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:11.484774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:11.487685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:11.489968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 377/732] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:11.496704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.591790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:11.641126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b9003-1ba9-492e-afce-cc3fbba296c3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.680602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.681687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.682090 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.682371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.683313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:11.683628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:11.685641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:11.687175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664289f8-78c6-49ef-a418-cb2aa3318e73 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.695774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 356/733] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:11.703656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.777945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.778711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.779075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.779433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.780396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:11.780793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:11.782404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:11.818043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3cc423-2be2-493f-b0e1-3934676341e7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:11.823513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 378/734] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:11.828835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.855384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:11.855952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:11.856233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:11.856506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:11.868734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:11.870479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:11.878100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:11.887018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:11.887883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:11.888748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:11.888748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:11.888748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:11.888748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:11.888748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 357/735] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:11.897715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:11.935014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.010580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2f3fc-916b-4c31-bc0c-a161d4ae3e6f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.077969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.078645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.078989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.079274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.106169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876f668-37b0-43e0-9153-86656c62cd5d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.191831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.192682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.192972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.193266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.193447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.193985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:12.194294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.194893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.195685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.225278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 358/736] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:12.230861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.241857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.291640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.310948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 379/737] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:11 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 03:22:12.316887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.392540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f95416-ea78-486b-bf9c-38fdd03067be None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.436513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.437970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.438601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.439098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.444231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe250a-0417-41f0-b337-57e614acf786 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.448693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.450434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-6c8af8b1-223a-4a16-bacd-bbbdd0de7cef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:12.465889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:12.466831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:12.467770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:12.471049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 359/738] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:22:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:12.473578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c77067-abec-419f-8c8b-8822bbadfb72 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.474137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c77067-abec-419f-8c8b-8822bbadfb72 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.474635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c77067-abec-419f-8c8b-8822bbadfb72 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.476351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 360/739] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:12.477275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c77067-abec-419f-8c8b-8822bbadfb72 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.492369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.493018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.493331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.493606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.503668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.506869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.508633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-2a98a311-3fd8-4a6b-ab0a-975e062da290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:12.524674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:12.525657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:12.526558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:12.526558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:12.526558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:12.526558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:12.528513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 380/740] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:12.529969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.530459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.530985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.554325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:12.555770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:12.590965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.605185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c77067-abec-419f-8c8b-8822bbadfb72 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.632766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 381/741] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 03:22:12.638282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.645847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.647663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.648101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.648588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.649680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:12.650142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.652412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.673722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 361/742] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Aug 4 03:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:12.675618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.778114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cf30b-17d8-4f9d-94f1-dc52e730020d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.801763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c3ace-8a8f-4339-8050-7663f09a9102 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:12.808762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.825017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.825912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.826357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.826780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.828153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:12.828893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.831258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.844006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d4bb5-270a-418f-808b-be4aafbc371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:12.844639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d4bb5-270a-418f-808b-be4aafbc371e None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.844924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d4bb5-270a-418f-808b-be4aafbc371e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.845223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d4bb5-270a-418f-808b-be4aafbc371e None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.845716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 382/743] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:12.846511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d4bb5-270a-418f-808b-be4aafbc371e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:12.846956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.848083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d4bb5-270a-418f-808b-be4aafbc371e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:12.922235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 362/744] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:12 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:12.923478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74ae4c-1e29-470f-8c0b-b142c471964f None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:12.923766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74ae4c-1e29-470f-8c0b-b142c471964f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:12.924037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74ae4c-1e29-470f-8c0b-b142c471964f None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:12.929710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 363/745] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:12.929710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74ae4c-1e29-470f-8c0b-b142c471964f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:12.944523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:12.967907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4ac55-e5b1-4d40-a0a8-824744785922 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.021136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff9e8e-e2eb-46fc-a15c-fdd0d4ffbd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.021972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff9e8e-e2eb-46fc-a15c-fdd0d4ffbd79 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.022373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff9e8e-e2eb-46fc-a15c-fdd0d4ffbd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.022795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff9e8e-e2eb-46fc-a15c-fdd0d4ffbd79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.083964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.108449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff9e8e-e2eb-46fc-a15c-fdd0d4ffbd79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.110243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff9e8e-e2eb-46fc-a15c-fdd0d4ffbd79 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.200154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74ae4c-1e29-470f-8c0b-b142c471964f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.234087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 383/746] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:13.235415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.235817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.236093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.241313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.242072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.242353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.242641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.249951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.251420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.257846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:13.259263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-44784bc5-8530-494b-9b38-e729a1acde73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:13.263784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:13.264621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:13.266132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 364/747] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:13.266602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592548a-c0f6-44eb-a32a-33dab8ba1039 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.266668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592548a-c0f6-44eb-a32a-33dab8ba1039 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.266913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592548a-c0f6-44eb-a32a-33dab8ba1039 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.268951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 365/748] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:13.271209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592548a-c0f6-44eb-a32a-33dab8ba1039 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.349236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 384/749] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:13.349236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.354781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.360599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.397138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592548a-c0f6-44eb-a32a-33dab8ba1039 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.436135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.436699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.436985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.437280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.448721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.450205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ef5c5921-f7ca-4269-8b2e-da56625cbee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:13.466790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:13.467657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:13.468550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:13.468550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:13.468550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:13.468550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:13.468550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 366/750] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:13.469271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.469368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.469629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.475936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a6e0b-75ae-4b36-8cb7-717370bfe6c8 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.488564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:13.489793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:13.519355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.519980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.520260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.520531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.521320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:13.521631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.523072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.540560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 385/751] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:13.540560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.567589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 367/752] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:13.568634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.625518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.635219 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.671154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238368-31db-45e6-a29a-e5a98926b5b8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.692419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229311-4ded-4cfb-9235-fc50b3e2be5f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.706543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.707194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.707481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.707873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.708514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.709925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:13.720852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:13.722340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:13.723167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 386/753] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:13.723986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.736036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.736620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.736951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.737341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.737950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:13.738253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.739656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.751666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 368/754] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:13.752356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.837957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd950f80-e538-4c07-b323-29afe9706b70 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.855151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839f3d3-3837-495e-b784-46cfae48906a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:13.865981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.869383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:13.874703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.878778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.878778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.878778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.878778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=d647c7fcd5fb490db5302a9ead7ab330) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:13.878778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:13.879177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:13.893779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 387/755] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/role_assignments?user.id=d647c7fcd5fb490db5302a9ead7ab330&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:13.893779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:13.902959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:13.903966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:13.904339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:13.904671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:13.918251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:13.923663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:13.924889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:13.925900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:13.931663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 369/756] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:13.931663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.034252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c068a-2244-42a7-a1e9-69caca537448 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.048073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e6650-345a-447e-8a25-d2b47de13fe6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.079600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.080421 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.080821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.081106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.081938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.082246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.083743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.088937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 388/757] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.089786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.101951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.103181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.103474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.103756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.115213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.117452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:14.134493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:14.135423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:14.136289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:14.137302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 370/758] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:14.138184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.175240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:14.190029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:14.226358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c1dd8-7c57-403f-8bd5-d3e64f93358e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.267140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.269046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.269549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.270147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.271395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.272174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.273937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.277122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b36568-7956-42bb-bdd2-502ed9927b18 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.290653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 389/759] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.290653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.326268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.328047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.328485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.328952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.329872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.330425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.332484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.340517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 371/760] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.341642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.405455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdaf5e-174c-4bec-8c50-c739ed4aab1c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.431969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:14.445035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.445856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.446376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.446738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.447614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.448008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.449601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.453992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:14.464297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ba05d-ad20-471c-b822-d6d045ee86b9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.466686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 390/761] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.467746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.506738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.508107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.508107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.508107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.520037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.521650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:14.536836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:14.537749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:14.538650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:14.538650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:14.538650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:14.538650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:14.542698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 372/762] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:14.543481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.583795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61d025-c780-4002-8b63-78896b135198 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.622434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.623456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.624024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.624503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.625505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] RBAC: Adding query filter params (interface=public, service_id=6b4f6b9d3d9e4ea59d92bbee5962bb31, region_id=RegionOne) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.626025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.627929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.641729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 391/763] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/endpoints?service_id=6b4f6b9d3d9e4ea59d92bbee5962bb31&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.643684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.654201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6b5e7-c303-4c4b-bc83-018bdbaad728 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.691318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.692219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.692355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.692568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.693402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=4444ac94fc184e949c285d365931d0f2) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.693722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.695275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.698881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:14.709367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 373/764] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/role_assignments?user.id=4444ac94fc184e949c285d365931d0f2&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.713844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.728564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:14.765180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ef04-66ac-4ec8-b8dc-57bfa8696653 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.807587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.808681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.809168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.809699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.810602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:14.811129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.812795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.829092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 392/765] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:14.842514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:14.931772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c9d9e-5a27-4a3d-854d-7fb5ac53a3eb None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:14.979388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991c3d-c5c3-4419-8d61-ded73b589bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:14.980095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991c3d-c5c3-4419-8d61-ded73b589bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:14.980375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991c3d-c5c3-4419-8d61-ded73b589bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:14.980646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991c3d-c5c3-4419-8d61-ded73b589bc7 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:14.981359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991c3d-c5c3-4419-8d61-ded73b589bc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:14.982865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991c3d-c5c3-4419-8d61-ded73b589bc7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:14.990302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 374/766] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:15.036999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11069bd6-8c65-4f19-8f12-17a2473bbcd5 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.037325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11069bd6-8c65-4f19-8f12-17a2473bbcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.037589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11069bd6-8c65-4f19-8f12-17a2473bbcd5 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.039275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 375/767] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:15.044817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.045159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.045386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.065852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:15.067199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:15.189939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf05b8e-3b3b-4bb7-9402-3ea9f3600013 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:15.200309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 376/768] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:15.201022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:15.224877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:15.227066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:15.239147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:15.240234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.240722 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.241228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.249928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:15.252652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:15.265517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:15.266476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:15.267380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 393/769] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:15.273132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:15.347726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33d08a-1f17-4bb3-9271-ec462c59e835 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:15.391826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:15.394082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.396351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.396776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] PATH_INFO: `/v3/services/network` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.397612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:15.400592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:15.422080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:15.423389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:15.424444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:15.431133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 377/770] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:15.438116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:15.478597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013196a4-b754-4cde-b785-e30c70f0d625 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:15.551445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:15.554897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.555334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.555756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.556703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:15.557962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:15.560618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:15.584337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:15.612998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 394/771] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:15.612998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:15.647382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c444e-bed6-4b76-b9ce-b27bfa906935 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:15.683217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:15.696984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:15.697820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.698196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.698595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.699372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] RBAC: Adding query filter params (name=network) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:15.699785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:15.701263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:15.707978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 378/772] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:15.717079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:15.829593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee43d8b-a533-46da-8c26-ea11ce6d0575 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:15.884396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e3dee-fb29-49d9-b7b0-95d6e9fdd408 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:15.916871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:15.923661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:15.925581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.925866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.926143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.933272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:15.933272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:15.933272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:15.933272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:15.933639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:15.933639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:15.933639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:15.947066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 379/773] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:15.948120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:15.950945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:15.952049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:15.983475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:15.984571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:15.985760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:15.985760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:15.985760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:15.985760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:15.985760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 395/774] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:15.991987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.072673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a3ebe-95d1-4f9a-83a2-bda7cfa655c9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.116521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.119208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.119488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.119762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.120539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] RBAC: Adding query filter params (interface=public, service_id=a884b421e6844e7891a92dfb3ddc0927, region_id=RegionOne) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:16.120843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.122254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.140192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 380/775] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/endpoints?service_id=a884b421e6844e7891a92dfb3ddc0927&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:16.148252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.200241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:16.229075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047b068-b60d-4b42-9df5-d12d4a28b53c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.247690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:16.267575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.268846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.269133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.269433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.270131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:16.270432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.271806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c206a-01be-419f-b5a0-5723687b1739 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.288078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 396/776] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:16.289458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644d6cb-f147-42db-a890-32e532d76368 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.290310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644d6cb-f147-42db-a890-32e532d76368 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.290599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644d6cb-f147-42db-a890-32e532d76368 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.296131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 397/777] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:16.296131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644d6cb-f147-42db-a890-32e532d76368 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.308211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233cdf03-f557-4e64-8d7f-9e3ec40c98c5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.355927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a223d-e31c-4732-bd68-2f3db1e1455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.356574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a223d-e31c-4732-bd68-2f3db1e1455b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.356821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a223d-e31c-4732-bd68-2f3db1e1455b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.357075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a223d-e31c-4732-bd68-2f3db1e1455b None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.357777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a223d-e31c-4732-bd68-2f3db1e1455b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.359164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a223d-e31c-4732-bd68-2f3db1e1455b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.373260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 381/778] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:16 2022] GET /identity/v3/services => generated 772 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:16.373260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.373260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.373260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.391487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:16.392757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:16.469910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 382/779] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:16.495049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644d6cb-f147-42db-a890-32e532d76368 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.535288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.550419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.553434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.553924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.554346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.565664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:16.569002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.570547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:16.591786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:16.592881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:16.593900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 398/780] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:16.599919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.626376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:16.682010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccd4bb-234a-41c0-b205-189dea112700 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.712623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d6d94-4097-4364-a6c4-e7bae2623485 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.720844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.722302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.722729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.723219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.734883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:16.740405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:16.741358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:16.742201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:16.745031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 383/781] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:16.750583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.751212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.751546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.751831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.752081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.752691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:16.753471 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.754936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.768824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 399/782] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:16.774030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:16.837895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:16.866641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9ed5f-9d3a-43f3-9f9b-c40339636bac None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.882913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:16.885918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fdbb1-e85d-4c79-b584-62c536ed5788 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:16.906691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.907679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.908174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.909370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.910336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:16.910863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.912639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.922995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 384/783] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:16.927719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a317b-859b-47a2-8a38-c9487e862b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:16.927719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a317b-859b-47a2-8a38-c9487e862b64 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:16.927719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a317b-859b-47a2-8a38-c9487e862b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:16.927719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a317b-859b-47a2-8a38-c9487e862b64 None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/09fbb54e522c47d89de166682742336f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:16.928066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a317b-859b-47a2-8a38-c9487e862b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=09fbb54e522c47d89de166682742336f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:16.929149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a317b-859b-47a2-8a38-c9487e862b64 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:16.929595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.013315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 400/784] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:16 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/09fbb54e522c47d89de166682742336f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:22:17.039910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bc7c5-4cec-4b95-bd88-6c325f86aede None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.077290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.078323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.078856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.079336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.091958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.093826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ec9fd669-5301-431d-b994-17aaff62a7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:17.110200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:17.111120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:17.112021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:17.112021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:17.112021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:17.112021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:17.113396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e766e6d-60fb-4561-a448-696a76d33d8d None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.113396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e766e6d-60fb-4561-a448-696a76d33d8d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.113575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e766e6d-60fb-4561-a448-696a76d33d8d None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.115276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 385/786] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:17.119034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 401/786] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:17.120613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.121139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.121649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.123904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e766e6d-60fb-4561-a448-696a76d33d8d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.142384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:17.144688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:17.168905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:17.229223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 386/787] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:17.252216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.256961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e766e6d-60fb-4561-a448-696a76d33d8d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.276783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:17.297800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.298813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.300607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.301102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.302205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:17.302751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.304492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:17.323129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 402/788] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:17.333719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.367964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cce31-5a3a-4ac7-92c9-58de3f327513 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.406999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04aaee-a99b-4d3f-b514-3cf47349866d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.407648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04aaee-a99b-4d3f-b514-3cf47349866d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.407927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04aaee-a99b-4d3f-b514-3cf47349866d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.408261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04aaee-a99b-4d3f-b514-3cf47349866d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.420111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04aaee-a99b-4d3f-b514-3cf47349866d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-fa04aaee-a99b-4d3f-b514-3cf47349866d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:17.425997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:17.426983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:17.428927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 387/789] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:17.432141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c37e4d-bd71-4073-bcac-bb1432904be7 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.432415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c37e4d-bd71-4073-bcac-bb1432904be7 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.432687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c37e4d-bd71-4073-bcac-bb1432904be7 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.434163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 388/790] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:17.437231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c37e4d-bd71-4073-bcac-bb1432904be7 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.452554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c752-12d4-4ec8-a97d-af813a6a81bb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.459038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:17.492335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.492983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.493286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.493558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.505035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.506527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5a41cfb6-4502-496e-a94b-950509906b05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:17.522399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:17.523447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:17.524358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:17.528258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 403/791] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:17.529462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.529757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.530028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.548803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:17.550691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:17.582039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c37e4d-bd71-4073-bcac-bb1432904be7 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.582945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:17.633221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 404/792] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:17.634887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.640555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.641074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.643591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.643870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.644609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:17.644918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.649056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:17.662069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 389/793] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:17.663037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.745228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:17.762081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea983a-3222-4a11-98a0-b8e43718bf13 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.769822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa2665-6113-4d1e-bf21-ae076d48c9d5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.803046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.803586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.803862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.804133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.804950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:17.805272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.806717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:17.816107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.818118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.818625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.819114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.819963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.821652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:17.831051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 405/794] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:17.831051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:17.839199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:17.840273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 03:22:17.841285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:17.845559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 390/795] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:17.847116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:17.865763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:17.946506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267861d-3f73-45b0-8341-02928c3456c0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.978274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1332769-9730-4adf-a377-7eb192727f9f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:17.981922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:17.982379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:17.982726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:17.983063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:17.993852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:17.995507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:18.011789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:18.012678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:18.013790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:18.013790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:18.013790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:18.013790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:18.013790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 406/796] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:18.014969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.022500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.023411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.023875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.024329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.037231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.039113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.041097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:18.069177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:18.070396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:18.071253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:18.071253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:18.071253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:18.071253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:18.071253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 391/797] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:18.071253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.103241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.126120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66aabb2-2e31-4bd9-9bc9-9567e35d71b3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.167376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.170926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.171312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.171653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.172409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:18.172776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.174592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.178823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78788433-ae19-4ec2-b984-ec809e4a6efd None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.184362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 407/798] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:18.185387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.217745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.218785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.219271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.219751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.220647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:18.221876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.223571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.237062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 392/799] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:18.238351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.290583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3536cf-d115-4358-9e60-cf57cf3d8ae8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.301805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.325596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.326104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.326382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.326654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.327406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:18.327645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.329228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.338937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.347601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 408/800] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:18.348484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.356978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3c39e-f2c6-4f1a-8cf7-5f73932fcf79 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.397774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.399176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.399725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.400206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.401177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:18.401806 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.403416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.409563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 393/801] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:18.410706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.460541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04892388-77b0-4b55-838e-b3aeb3497865 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.495492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.496134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.496411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.496680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.505110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.506585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:18.519368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:18.520355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 409/802] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:18.521900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.531085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f2d87-04cf-4dd9-9cc3-394d9caec1db None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.551520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.571866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.572898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.573451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.573947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.581368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.584040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.589607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:18.597625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:18.598499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:18.599376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:18.601919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 394/803] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:18.609747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.630285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784d75b-beb6-4565-b36e-462b1f85f0f2 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.664944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.665610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.665871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.666141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.666852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.668218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.714066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e92656-9f5b-4689-a87f-2d340ae14d7a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.752755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 410/804] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:18.754774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.779455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.780549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.780996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.781298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.782201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:18.782512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.784671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb94ca-cc5d-410c-beab-7c819e27a720 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.810414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 395/805] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:18.812219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd7ded-74bb-4659-978a-24e65ff137f2 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.812527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd7ded-74bb-4659-978a-24e65ff137f2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.812809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd7ded-74bb-4659-978a-24e65ff137f2 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.819894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 396/806] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:18.819894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd7ded-74bb-4659-978a-24e65ff137f2 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:18.862995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.896933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd909d-fbb0-4f7a-aaeb-d90d51c711f4 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.916314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:18.940854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:18.941681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.942075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.942450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.944425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:18.944858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:18.946369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab17352-6cf0-416b-b40d-49ff7371ab18 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:18.966914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 411/807] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:18.966914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:18.966914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:18.966914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:18.967301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd7ded-74bb-4659-978a-24e65ff137f2 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:18.985069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:18.986301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:19.013268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.014602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.014602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.014602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.034125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.036558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:19.060237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:19.061155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:19.062071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:19.062071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:19.062071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:19.062071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:19.063220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 397/808] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:19.064022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.067248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 412/809] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:19.074345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.182430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:19.201737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96621f8-da82-4eeb-b6cb-7b990e9e97b9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.205913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f4013-2348-4042-9232-d48132f0ce6e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.210633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:19.239368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.239988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.240271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.240595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.241334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:19.241640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.243021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.254830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.256070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.256600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.257078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/d647c7fcd5fb490db5302a9ead7ab330/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.257368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 413/810] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:19.258212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.260035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=d647c7fcd5fb490db5302a9ead7ab330, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.263198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.342052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 398/811] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:19 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/d647c7fcd5fb490db5302a9ead7ab330/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:22:19.344574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.385059 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2397e-4fab-4c09-adc5-4b051ac3551d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.428044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.428740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.429021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.429324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] PATH_INFO: `/v3/services/network` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.429961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.431367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-01f87611-619b-4ce3-b3d9-4dca23bf6177 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:19.442050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:19.443148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:19.444420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 414/812] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:19.445628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2784a9c-5776-4f9e-b733-b307fcb8ee33 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.445628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2784a9c-5776-4f9e-b733-b307fcb8ee33 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.445628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2784a9c-5776-4f9e-b733-b307fcb8ee33 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.451028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 415/813] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:19.451942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2784a9c-5776-4f9e-b733-b307fcb8ee33 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.478147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be6778-30cb-470d-8042-5cb7017d052f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.490517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:19.530754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.531424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.531796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.532072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.541418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.543002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-45b5143d-006e-4ef0-ba17-fc54cb892fb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:19.557536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:19.558459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:19.559799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 399/814] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:19.560451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.560852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.560961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.568469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:19.580936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:19.586422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:19.586536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2784a9c-5776-4f9e-b733-b307fcb8ee33 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.625905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.628494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.628961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.629471 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.630331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] RBAC: Adding query filter params (name=network) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:19.630833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.632837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.639153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 416/815] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:19.640161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.667957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 400/816] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:19.668809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.759972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:19.769864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10688221-b878-410d-b9ba-9db55346f793 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.807448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.808140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.808664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.808819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.809587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:19.809886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.811435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.816163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dec75c-6854-4016-9b6d-8a5d23f0f16f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.825137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 417/817] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:19.825829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.859983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.861517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.862520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.862974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.864010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] RBAC: Adding query filter params (type=network) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:19.865303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:19.866875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:19.868406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:19.873860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 401/818] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:19.874833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:19.947563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4d8a7-9069-476d-b68b-267b55030e10 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:19.984085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:19.984738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:19.985016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:19.985326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:19.996636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:20.001931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:20.004163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:20.004970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 418/819] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:20.005657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:20.012432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93c304-6323-49ec-ab4b-e0d33d2a9ccd None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:20.021105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:20.050344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:20.051416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.051875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.052373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/4444ac94fc184e949c285d365931d0f2/roles/b94a012ed1954740858f831257bb8811` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.070830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=4444ac94fc184e949c285d365931d0f2, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:20.072572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:20.108285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78605-712f-4f0c-be37-56a623acbc22 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:20.139241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 402/820] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:19 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/4444ac94fc184e949c285d365931d0f2/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:22:20.140444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:20.143989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:20.144677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.145414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.145414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.146833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:20.147449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:20.200715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:20.234032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 419/821] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:20 2022] POST /identity/v3/endpoints => generated 337 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:20.261995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f3541-3c15-4203-a17d-b0e2b4d7ec5c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:20.310426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:20.311421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.311901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.312371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.313467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:20.313997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:20.316807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:20.327024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 403/822] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:20.333921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:20.419248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:20.454261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e38396-3076-4336-9626-9e5c6bc3fa60 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:20.497317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:20.498341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.498824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.499298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.514678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:20.516639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:20.534718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:20.535656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:20.536512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:20.536512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:20.536512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:20.536512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:20.538719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 420/823] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:20.544021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:20.662675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20cff4-66f9-473b-beef-a3b3a37fb92c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:20.671887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:20.703447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:20.704266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.704855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.705126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.706868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:20.707171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:20.708632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:20.723694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 404/824] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:20.729607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:20.848161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ce5be-f2d3-46cc-b77b-2fd7979d7978 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:20.871747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:20.884442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:20.887584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.887584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.887584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.893986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:20.895638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-912c4613-de2d-44d7-bbbe-e5be23eccdb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:20.907384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:20.908893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:20.910048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 421/825] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:20.915152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:20.994673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2bae8-bb45-49dd-bbe1-f7967b68239e None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:20.995043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2bae8-bb45-49dd-bbe1-f7967b68239e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:20.995374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2bae8-bb45-49dd-bbe1-f7967b68239e None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:20.997045 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 422/826] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:21.001573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.001924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.002306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.022529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:21.023829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:21.032689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e2d02-e99c-4ba5-a6e6-c246607da30c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.076531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.078003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.078744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.079196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.081128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:21.082088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.083736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.100269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 423/827] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:21.106308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 405/828] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:21.112063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.122285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.156349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:21.225493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68275aeb-3c86-4ad1-adde-a1f228226c09 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.243188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:21.252971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327864-3ff4-41a2-9a0a-d8e868aeb39e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.272490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.273125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.273435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.273710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.293135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.294311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.294798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.295271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.296185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.297921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:21.308056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:21.309043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:21.309876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:21.317703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 406/829] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:21.318811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.361473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.364176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.424263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:21.437587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcc3eb-1c68-4471-8d9e-faa77823c1e2 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.491160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.491788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.492068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.492343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.493037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:21.493382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.494822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.495055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 424/830] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 03:22:21.501254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 407/831] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:21.502953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.507263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.520881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:21.633008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636daa0-e1c9-429e-916f-9edbb55a70c6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.634846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff0cd72-a02a-4240-9858-ed51338c0f3c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.676679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.677342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.677676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.678033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.678544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:21.678839 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.679153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.679800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.680077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.680230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29152de2-5bd0-470d-aa43-b33c35a1c906 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.680378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.697596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 408/832] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:21.697596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0623de-eb0e-474a-bbd8-3ceb8e688f67 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.697596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0623de-eb0e-474a-bbd8-3ceb8e688f67 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.697596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0623de-eb0e-474a-bbd8-3ceb8e688f67 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.697596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 409/833] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:21.697596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.698076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.701768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.706943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.708433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.728448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:21.729713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:21.731274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:21.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:21.733642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:21.733642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:21.733642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:21.733642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:21.734617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 425/834] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:21.742161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.748487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:21.830838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:21.842907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 410/835] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:21.870396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:21.897544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda97244-0f7c-4345-8588-e685a10089c0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:21.941157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:21.941891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:21.942173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:21.942450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:21.943151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:21.943456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:21.944859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:21.960149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 426/836] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:21.966243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.007523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361b9ad-d625-46af-abbd-2a3eafe4e785 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.052218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.052868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.053148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.053476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.067541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:22.075236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:22.076190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:22.077018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:22.082897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 411/837] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:22.084042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.096519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:22.118125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9875a0-f84c-4a4b-846e-3af60a780cd9 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.152929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:22.180306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.181440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.181922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.183366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.197876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.199914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:22.215012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:22.215910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:22.216794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:22.219414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 427/838] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:22.230112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.230297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e9e15-8f29-4183-b322-383b8452a458 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.278035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.279037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.279316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.279583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.281586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:22.281898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.283858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.297240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 412/839] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:22.304773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.360650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf5da-6fe2-40cf-a604-3b2cbfd31881 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.404640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:22.409816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.412912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.413227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.413525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.414359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:22.414672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.416482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.436412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 428/840] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:22.437121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.448477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fd990-6a03-445e-b65d-bca1afb6d633 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.488158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:22.494740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.495384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.496054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.496054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.508798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.510512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-32b08a83-b609-4271-97ea-349d2fc215f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:22.525891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:22.526857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:22.527799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:22.527799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:22.527799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:22.527799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:22.527799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 413/841] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:22.528880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec76cd-7500-492e-843d-fac6c70342b9 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.528880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec76cd-7500-492e-843d-fac6c70342b9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.528880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec76cd-7500-492e-843d-fac6c70342b9 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.530338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 414/842] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:22.532459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec76cd-7500-492e-843d-fac6c70342b9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.554853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21867350-b914-4e7d-96ef-2fc6e9798bcb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.594867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.596050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.596548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.596831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.597683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=09fbb54e522c47d89de166682742336f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:22.597992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.599529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21dbe4-50e3-4ed3-a7a7-d75468deb749 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.605817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 429/843] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/role_assignments?user.id=09fbb54e522c47d89de166682742336f&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:22.607031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.607311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.607582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.638141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:22.639259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:22.664500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec76cd-7500-492e-843d-fac6c70342b9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.698114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:22.720356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.721976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.722617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.723212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 430/844] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:22.723678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.724469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:22.724776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.733260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.742607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 415/845] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:22.747340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.748828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.801632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:22.874631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07c325-952c-4604-9fbe-982fd7c6e0dd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.875052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd798c-be5c-4a50-b967-ba0eb469c1b5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:22.914075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.914716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.915009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.915261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.917501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:22.919011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:22.919502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:22.919980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:22.922953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:22.924765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:22.933772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:22.938163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:22.939142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:22.940099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 416/846] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:22.942743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:22.943644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:22.944436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 431/847] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:22.946551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.951780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:22.999696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.065381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.068914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b042e8f-efe1-4d7e-9e86-690fda70ef1d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.070962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4798160-9846-4b58-bfd5-26405392b4b9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.103486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.103990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.104473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.105041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.106238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:23.106670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.108186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.111452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.112265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.112696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.113116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.114200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:23.114717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.116362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.122007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 432/848] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:23.123077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.144544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 417/849] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Aug 4 03:22:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:23.152241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.246544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51d135-6a4b-4295-b636-129391747f5c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.274197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbb9bd-dd90-49e1-9667-4c4b72a72c7e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.277523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.288265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.289230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.289540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.289817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.303245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.304973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.320058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.321054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.321580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:23.321929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:23.322829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:23.323712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:23.323712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:23.323712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:23.323712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:23.323712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 433/850] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:23.324616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.328563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.357629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.425268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.427925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.451375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190b412-fe70-4bff-9496-7c9a59b49594 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.489274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.489977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.490327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.490665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.491433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:23.491855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.493337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.511299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 434/851] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:23.512290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.535451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.545349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 418/852] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:23.546428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.588532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.620531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9deed-65eb-4e64-949d-52946b9d6757 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.650741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e81225-041e-447b-a12d-04aed4f2ca65 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.660259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.660968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.661331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.661615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.669274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.670739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:23.683055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:23.684007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:23.684897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 435/853] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:23.690124 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.693482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.694689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.695168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.695672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.707603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.709427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:23.725496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:23.726380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:23.727250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:23.727250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:23.727250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:23.727250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:23.729115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 419/854] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:23.734749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.788261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.808924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf844e-515e-42c7-883d-fd5dc4d41f67 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.842063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9742-0dfe-4053-8c3a-bff143bdc1cf None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:23.843729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:23.850242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.850866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.851147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.851419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.852215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:23.852535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.854058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.876663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 436/855] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:23.878516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.880180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:23.880815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.881095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.881399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.882163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:23.882466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:23.883970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0cbbd-0401-4d5f-a08b-1b7c1fc387ab None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:23.897326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 420/856] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:23.903078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636703f-9ddb-4673-8afd-6c3ec8cf5760 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:23.903360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636703f-9ddb-4673-8afd-6c3ec8cf5760 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:23.903630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636703f-9ddb-4673-8afd-6c3ec8cf5760 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:23.905017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 421/857] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:23.909647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636703f-9ddb-4673-8afd-6c3ec8cf5760 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:23.983784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c5a6f-bd05-4296-9da6-0801d27b0bab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.016711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636703f-9ddb-4673-8afd-6c3ec8cf5760 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.019178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.019910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.020185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.020464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.028214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.051242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.051809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.052089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.052363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.059584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.061028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-9b3bd459-158d-42fe-acef-25f623c2a5aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:24.072758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:24.073630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:24.074487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 422/858] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:24.075295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.075295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.075557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.086366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.094471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:24.100309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:24.100432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.100432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.181829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 423/859] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:24.181829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.209536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 437/860] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:24.214330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.284647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.287375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d533-5730-41bc-b6ac-4d3e71a9eb25 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.309651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.320293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03dad5-c590-4a97-841a-acf620efe4f3 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.325357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.325357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.325357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.325357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.325682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:24.325682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.327067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.346466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 424/861] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:24.346466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.363881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.369339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.369339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.369339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.369339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.369339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.444252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d41b3-6478-40b6-8eda-0b980c0ed548 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.470705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 438/862] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:24 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:24.472158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.497333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.499116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.499918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.500400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.519057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.521037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:24.543192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:24.544123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:24.544977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:24.544977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:24.544977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:24.544977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:24.551749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 425/863] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:24.553568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.567306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.613527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09624843-96ca-43f2-8292-fce2862e3055 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.657727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.659810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.660157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.660506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.661403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=d647c7fcd5fb490db5302a9ead7ab330) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:24.661783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.663329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.670502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 439/864] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/role_assignments?user.id=d647c7fcd5fb490db5302a9ead7ab330&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:24.679052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5292dcc-f518-4d53-a882-96165678d446 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.723291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.723996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.724274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.724548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.725291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:24.725598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.727070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706e029-9e6d-4794-a01c-804b5f0b76e3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.740985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 426/865] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:24.746882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.766987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6f2fb-927c-45a8-abe7-bd09188cfeed None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.767277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6f2fb-927c-45a8-abe7-bd09188cfeed None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.767555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6f2fb-927c-45a8-abe7-bd09188cfeed None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.769926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.773346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 427/866] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:24.775610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.775899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.776172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.794762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:24.795994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:24.874934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa0fc-f29f-4ade-9e6f-92fdeeaeea20 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:24.886746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:24.896008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 428/867] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:24.918851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:24.920986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:24.921866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:24.922154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:24.922431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:24.931374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:24.932835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:24.945146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:24.946081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:24.946928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 440/868] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:24.952173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:25.028062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c0fa-51e8-4145-8a72-6a6776903957 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:25.045171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:25.063921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:25.074550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:25.074550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:25.074550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:25.074550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:25.074550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:25.075195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7273c-c45d-4aa3-8dd8-58081aa5ee50 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:25.076947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:25.077985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:25.079004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 429/869] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:25.084034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:25.119092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:25.120080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:25.120656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:25.121163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:25.122318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:25.122866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:25.124631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:25.138903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 441/870] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:25.149313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:25.171930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:25.209595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676cfa6-bf3a-4ad9-be21-5fb02f68c073 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:25.255685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc0983-bbf0-467b-9280-9231a72b8bf7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:25.256768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:25.257803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:25.259001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:25.259458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:25.260299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:25.260749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:25.262354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:25.269350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 430/871] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:25.280481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:25.297756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:25.298386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:25.299166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:25.299451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:25.300251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=4444ac94fc184e949c285d365931d0f2) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:25.300565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:25.302046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f5167-618f-4e1f-b4b9-265585a24bb3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:25.304796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:25.323899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 442/872] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:25 2022] GET /identity/v3/role_assignments?user.id=4444ac94fc184e949c285d365931d0f2&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:25.395911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37c844-350a-4d91-b48a-fde0c7842802 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:25.427945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:25.437258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:25.437894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:25.438170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:25.438437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:25.439127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:25.440057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:25.441719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83464c-d1bb-46b9-8b5d-a54750b50f2d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:25.447529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 431/873] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:26.638044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6e560-f99e-4c96-8e8b-8669ac1ee7ae None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:26.638591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6e560-f99e-4c96-8e8b-8669ac1ee7ae None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:26.638981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6e560-f99e-4c96-8e8b-8669ac1ee7ae None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:26.640743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 443/874] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:26.646594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9fca-1b5d-47c1-b6a1-53fb7864599f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:26.647044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9fca-1b5d-47c1-b6a1-53fb7864599f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:26.647343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a9fca-1b5d-47c1-b6a1-53fb7864599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:26.666168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8d0a9fca-1b5d-47c1-b6a1-53fb7864599f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:26.667364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8d0a9fca-1b5d-47c1-b6a1-53fb7864599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:26.741412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 432/875] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:26 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:26.760465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6e560-f99e-4c96-8e8b-8669ac1ee7ae None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:26.823502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64047-ce54-4c5b-ad00-3bcfb3b48ef4 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:26.823770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64047-ce54-4c5b-ad00-3bcfb3b48ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:26.824050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e64047-ce54-4c5b-ad00-3bcfb3b48ef4 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:26.825668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 433/876] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:26.831536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:26.832235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:26.832611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:26.854454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:26.855906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:26.876336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6e560-f99e-4c96-8e8b-8669ac1ee7ae None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:26.920699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:26.921494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:26.921996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:26.922279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:26.923086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:26.925325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:26.934466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 434/877] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:26 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:26.943453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:26.944681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:26.945726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 444/878] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:26.950941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:26.966106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.007101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.033053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.062275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20be1a-2df1-43b4-a821-c3a118e40898 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.075925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ec846-242e-457f-91f2-bb1a00cb0a6b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.101456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.102216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.102722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.103064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.103926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:27.104296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.105905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.115456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.116226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.116564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 435/879] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:27.116815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.117174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.117954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.118339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.119459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:27.130559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:27.131575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:27.132664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:27.137039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 445/880] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:27.137976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.234448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ac102-b229-45ea-b795-5e0119084c95 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.250942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.261165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cdc8c-2288-453e-b5e4-609f7a996d51 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.274453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.275389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.275924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.276527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.277462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:27.277970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.279679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.291017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.291349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 436/881] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:27.293449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.304353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.305710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.306511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.307149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.308181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:27.308732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.311007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.318167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 446/882] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:27.323703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.431980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694cb07-370e-429c-88d7-2224d0752759 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.436990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c3dd0-5743-45e0-9cc3-b5b2c795cd08 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.473110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.473631 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.473774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.474060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.474329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.475014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.475120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:27.475348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.475788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.476254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.476764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.477440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] RBAC: Adding query filter params (interface=public, service_id=f320234a547d4f959a5ffe563f069866, region_id=RegionOne) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:27.477969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.479546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e14404-a0e5-405f-a8c4-f9830713bd52 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.489501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 437/883] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/endpoints?service_id=f320234a547d4f959a5ffe563f069866&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:27.489692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 447/884] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:27.491084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4a87-ff24-4dbc-ad9f-6802ec019660 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.491084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4a87-ff24-4dbc-ad9f-6802ec019660 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.491392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4a87-ff24-4dbc-ad9f-6802ec019660 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.492964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.493393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 438/885] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:27.498998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09670263-a51e-4d6b-82ea-c119d6cf3e64 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.499306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09670263-a51e-4d6b-82ea-c119d6cf3e64 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.499550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09670263-a51e-4d6b-82ea-c119d6cf3e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.521180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-09670263-a51e-4d6b-82ea-c119d6cf3e64 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:27.522441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-09670263-a51e-4d6b-82ea-c119d6cf3e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:27.564024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.564325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.609248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 439/886] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:27 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:27.609248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521a3cf-cdfc-4cad-8bd8-44c6206d44b9 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.609248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521a3cf-cdfc-4cad-8bd8-44c6206d44b9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.609248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521a3cf-cdfc-4cad-8bd8-44c6206d44b9 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.609248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 440/887] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:27.611989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.612662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.612939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.615182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5be9f9-8d9f-469f-ba09-c61ccb6e21a8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.637157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:27.638403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:27.661170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.661827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.662092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.662360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.663017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.664399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.670183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 448/888] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/services => generated 1011 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:22:27.671181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.721286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 441/889] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:27 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:27.744855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.792779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ceb68-2343-40cd-8f8d-5038329d5535 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.813589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.839864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.841127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.841613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.841890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.858281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:27.863795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:27.864712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:27.865621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:27.869306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 449/890] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:27.871019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:27.883438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7e737-dcf9-4baa-aeb0-34c4d1ac5692 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:27.885863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:27.926229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:27.926923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:27.927206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:27.927480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:27.928330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:27.930095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:27.994501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570939e-5f5b-4e57-9b54-ec55fb010504 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:28.037927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 442/891] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:27 2022] POST /identity/v3/services => generated 269 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:22:28.043658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:28.044415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:28.044768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:28.045073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:28.045913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:28.046329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:28.047775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:28.059301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 450/892] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:22:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:28.066033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:28.180382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f223c-04cd-41fc-9bed-b71f66b8bf7a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:28.220385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:28.221869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:28.222295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:28.222574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:28.237434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:28.238909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:28.254569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:28.255448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:28.256299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:28.256299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:28.256299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:28.256299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:28.256299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 443/893] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:28.262549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:28.302994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:28.370988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a389-3a3f-4d3f-9625-495c9ff337ac None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:28.407234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:28.408052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:28.408923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:28.409455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:28.410203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:28.410512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:28.411895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:28.426866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 451/894] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:28.432617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:28.451246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:28.539343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfd485-6921-49cf-bb51-ab732badc52b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:28.574232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:28.575243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:28.575725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:28.576196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:28.584801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:28.586497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:28.598229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:28.599149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:28.600050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:28.602611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 444/895] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:22:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:28.608536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:28.713009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:28.719906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ab74-9d7f-4dfb-9bcb-6307f72cf2b8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:28.756826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:28.757479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:28.757761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:28.758034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:28.758899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:28.759206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:28.760715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:28.779668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 452/896] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Aug 4 03:22:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:28.786010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:28.864173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:28.890179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b856-ce7b-4dae-afd7-f21310084161 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:28.933068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:28.933771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:28.934047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:28.934315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.004940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.006441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.080628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:29.106146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 445/897] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:28 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:29.111462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:29.217966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b4af-8808-4097-baeb-a525eb0ab8e5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:29.254919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:29.255671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.255955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.256232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.268436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.269946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:29.285644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:29.286641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:29.287894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:29.287894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:29.287894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:29.287894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:29.287894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 453/898] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:29.292647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:29.394070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bfba2-72ce-4c4e-99bd-5e4d6db93769 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:29.430991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:29.431530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.431810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.432126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.432813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:29.433109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.434496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607803a-e1e0-460c-a7d6-064b11d82dc3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.446523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 446/899] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:29.451701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:29.462091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:29.559976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6163b07-1532-4681-8074-24d81dbc3287 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:29.595716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:29.596368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.596649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.596922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.603962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.605447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:29.618070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:29.618987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:29.620246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 454/900] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:29.621857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d61c2-d473-4a44-a2d5-9ecda5dea0bc None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.622360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d61c2-d473-4a44-a2d5-9ecda5dea0bc None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.622833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d61c2-d473-4a44-a2d5-9ecda5dea0bc None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.628959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:29.629324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 447/901] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:29.631114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.631606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.632096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.651908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:29.653501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:29.682942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:29.696395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:29.735145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65ce9-25e9-48b9-a603-96bc35c44020 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:29.735973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 448/902] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:29.763293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:29.780664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:29.781660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.782146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.782619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.784266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:29.784800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.786546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.805712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 455/903] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:29.806905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:29.871467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544e3b1-5012-4405-9bbd-e2e7f9e39d1b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:29.914581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b184501-d714-48d4-a5c9-2f311beba42c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:29.920735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:29.921380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.921656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.921918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.922619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.924099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.941639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:29.950032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:29.954853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.954853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.954853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.954853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] RBAC: Adding query filter params (role.id=841751d723ec454aae186b4da332a393, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=4444ac94fc184e949c285d365931d0f2) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:29.954853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:29.954853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1543e7-0616-4dc4-ae87-40321b4201a9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:29.959963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 456/904] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:29 2022] GET /identity/v3/role_assignments?user.id=4444ac94fc184e949c285d365931d0f2&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=841751d723ec454aae186b4da332a393 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:29.961090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260cbc3-6acd-4139-bc18-433d84d6bc34 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.961414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260cbc3-6acd-4139-bc18-433d84d6bc34 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.961696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260cbc3-6acd-4139-bc18-433d84d6bc34 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.963060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 457/905] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:29.968782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199fe29-7312-45ec-81f3-144f6346e6cd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:29.969179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199fe29-7312-45ec-81f3-144f6346e6cd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:29.969490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199fe29-7312-45ec-81f3-144f6346e6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:29.989637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-5199fe29-7312-45ec-81f3-144f6346e6cd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:29.990903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5199fe29-7312-45ec-81f3-144f6346e6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:30.016664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 449/906] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:29 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:22:30.087065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 458/907] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:30.109915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.140629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:30.172034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa15e-8be5-4d02-acd5-9137156568d0 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.172357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa15e-8be5-4d02-acd5-9137156568d0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.172584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caa15e-8be5-4d02-acd5-9137156568d0 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.175246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 459/908] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:30.180840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.181422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.181913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.205385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:30.206091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:30.247473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2f714-1f70-412f-b9f4-4f4356b71d07 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.290211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 460/909] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:30.299034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.299788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.300141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.300481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.301273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.302772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.311784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:30.312112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:30.313023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:30.313858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 450/910] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:30.323323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.377105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:30.435799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:30.447013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487cec3-22fb-4eb9-85db-47f922bf1f7b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.449774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a1b-1d8c-49f1-be84-59ad87a02d19 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.491331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.491626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.491907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.492237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.492511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.492942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.493146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.493549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.493964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.494542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.494968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:30.495415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.497472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.504259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 451/911] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:30.505944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:30.506855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:30.507962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 461/912] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:30.510080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.513014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.633447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95162569-8538-41f4-86fe-86ca74fb0dd4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.647569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99631345-6572-4af0-9eb1-c4f8b7e91729 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.659942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:30.670438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.675521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.675521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.675521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.675521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:30.675521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.675521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.681245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 452/913] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:30.686386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.687585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.688457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.689111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.690152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.692128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:30.692673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.694376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.700310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 462/914] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:30.705823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.706754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:30.796276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8659b95-1a99-46f1-89f8-237755ad7e94 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.821674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e07175-59c1-4e4b-a535-b35e1c28fb82 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:30.832863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.833649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.833763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.834041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.834761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.836194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.861041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:30.862429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:30.862779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:30.863122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:30.863877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:30.864253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:30.865783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:30.873283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 463/915] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:30.877114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:30.907242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 453/916] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:30.933268 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:31.008896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f611d17-cd6f-47ac-a72c-8e1f037f3e3c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:31.055757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:31.056396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:31.056681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:31.057006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:31.057809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] RBAC: Adding query filter params (interface=public, service_id=011cfb22f6eb4780ba874b4f8cb49aeb, region_id=RegionOne) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:31.058121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:31.059575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47362e-0518-4b31-ab83-1e1752b9efee None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:31.066550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 464/917] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:30 2022] GET /identity/v3/endpoints?service_id=011cfb22f6eb4780ba874b4f8cb49aeb&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:31.072052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:31.149741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:31.193451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ef71-5419-474f-b9ea-d092cacb5e7a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:31.231262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2c236-ea97-4adb-80dc-c12c512dca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:31.232135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2c236-ea97-4adb-80dc-c12c512dca38 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:31.232590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2c236-ea97-4adb-80dc-c12c512dca38 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:31.233022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2c236-ea97-4adb-80dc-c12c512dca38 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:31.233859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2c236-ea97-4adb-80dc-c12c512dca38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:31.235441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2c236-ea97-4adb-80dc-c12c512dca38 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:31.243437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 454/918] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:31 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:22:32.063399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185650d-11cf-463a-842b-0383899ad083 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.063914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185650d-11cf-463a-842b-0383899ad083 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.063990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9185650d-11cf-463a-842b-0383899ad083 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.066311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 465/919] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:32.071135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26494d-7e12-4ca8-b688-7a0ec3a95679 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.071543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26494d-7e12-4ca8-b688-7a0ec3a95679 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.071709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26494d-7e12-4ca8-b688-7a0ec3a95679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.091633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7a26494d-7e12-4ca8-b688-7a0ec3a95679 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:32.092843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7a26494d-7e12-4ca8-b688-7a0ec3a95679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:32.170836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 455/920] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:32.192015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185650d-11cf-463a-842b-0383899ad083 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.198274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17d93e-25fd-4775-baa6-b6ca790c0181 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.198562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17d93e-25fd-4775-baa6-b6ca790c0181 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.198834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17d93e-25fd-4775-baa6-b6ca790c0181 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.200420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 456/921] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:32.205228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.205643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.205918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.224885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:32.226162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:32.290624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:32.305282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 457/922] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:32.323689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.328005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:32.335382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9185650d-11cf-463a-842b-0383899ad083 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.376595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.378735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.379228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.379736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.393092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:32.399287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:32.400253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:32.401062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:32.403925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 466/923] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:32.412523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.430419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891efdf-7d2c-4530-a6dd-d6b32c31a7d0 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.466432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.467075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.467354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.467624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.468255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:32.470112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:32.481785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:32.482856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:32.483670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:32.484764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 458/924] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:32.490017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.531955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63a41-250f-45e2-b133-8664bd8e9d3d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.568721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.569423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.569707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.569984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.570760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:32.571048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:32.572503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:32.575963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:32.585435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 467/925] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:32.587684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.613777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:32.638445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab1caa-8de3-4071-a66a-f82e7c6f7000 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.684212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.685642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.686159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.686880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.687796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:32.688519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:32.691445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:32.702897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 459/926] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:32.703932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.716389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76032d-bad6-4bbc-8055-d19f066c1544 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.762083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.762976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.763355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.763940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.775998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:32.777499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:32.793500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:32.794604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:32.795677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:32.795677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:32.795677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:32.795677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:32.801474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 468/927] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:32.803643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.826640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de32d-5f36-4789-8ba4-96740ea0514c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.854881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:32.866734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.867528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.867866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.868197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.869060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:32.869501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:32.870976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:32.881490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 460/928] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:32.882761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:32.899901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:32.934613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5d946-06f7-4833-b032-e4cbf4169e01 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:32.980750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:32.981844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:32.982694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:32.983146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:32.984252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:32.984718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:32.986971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.003484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b019f57-6214-4bca-8cb1-6359791f3b92 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.004753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 469/929] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:33.010817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.050176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.051045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.051340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.051613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.052386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] RBAC: Adding query filter params (interface=public, service_id=f97517b623c048d1a864d94424112e99, region_id=RegionOne) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:33.053624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.055027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4c0dd-1f15-4089-9445-aae183c314fc None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.065366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 461/930] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:32 2022] GET /identity/v3/endpoints?service_id=f97517b623c048d1a864d94424112e99&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:33.065366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b448d-67e2-4944-8b2a-5c429398dbcc None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.065366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b448d-67e2-4944-8b2a-5c429398dbcc None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.065366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b448d-67e2-4944-8b2a-5c429398dbcc None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.065366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 462/931] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:33.069049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b448d-67e2-4944-8b2a-5c429398dbcc None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.129289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.133864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd8a99-ecbb-4562-b038-41f1b2bdd730 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.167943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.173110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.174602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.174900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.175174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.183736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.185222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.187101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b448d-67e2-4944-8b2a-5c429398dbcc None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-91bb25a8-ddce-4a5a-8799-b19881efd29f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:33.198599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:33.199800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 470/932] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4fa4b-cb63-4f19-af41-289ab8023899 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.205935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4fa4b-cb63-4f19-af41-289ab8023899 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.206782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4fa4b-cb63-4f19-af41-289ab8023899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.222546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1bd4fa4b-cb63-4f19-af41-289ab8023899 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:33.223860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1bd4fa4b-cb63-4f19-af41-289ab8023899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:33.231827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.232459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.232737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.233006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.234079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.235757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.249577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 463/933] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:22:33.250456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.313868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 471/934] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:33.313868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2969f-991b-4398-9d4b-ef80061e19d4 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.313868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2969f-991b-4398-9d4b-ef80061e19d4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.313868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2969f-991b-4398-9d4b-ef80061e19d4 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.313868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 472/935] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:33.317039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.317356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.317630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.338782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:33.340002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:33.387735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5416a0-4270-4932-901a-0834b93b87cd None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.406995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.429237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 473/936] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:33.429237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.430229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.431003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.431137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.431402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.432212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:33.432520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.434104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.452338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 464/937] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:33.453168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.488829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.540016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c846f-e469-4ebe-b9ba-7a9207487d57 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.563108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e88c3-0801-453f-b76c-bf44f344f190 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.575734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.576768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.577172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.577451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.578044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.579482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.582751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING keystone.server.flask.common [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 03:22:33.601736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.602377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.602656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.602925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.603549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.604958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:33.614042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:33.614956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:33.615781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 465/938] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:33.616714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.630054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.sql.core [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 03:22:33.630054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 03:22:33.630054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [parameters: {'id': '8ac512ddb00b4efd9ddeb36a84f6d47d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 03:22:33.630054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80714) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8ac512ddb00b4efd9ddeb36a84f6d47d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8ac512ddb00b4efd9ddeb36a84f6d47d. Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application conn.query(q) Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 03:22:33.640259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result.read() Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 03:22:33.641023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application next(self.gen) Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application yield resource Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application next(self.gen) Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise self.value Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application session.commit() Aug 04 03:22:33.642011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.session.flush() Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application compat.raise_( Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 03:22:33.644156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 03:22:33.645037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application conn.query(q) Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application result.read() Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 03:22:33.646836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [parameters: {'id': '8ac512ddb00b4efd9ddeb36a84f6d47d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.648791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:33.650796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8ac512ddb00b4efd9ddeb36a84f6d47d. Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 474/939] 10.222.0.66 () {60 vars in 1140 bytes} [Thu Aug 4 03:22:33 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:33.652780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.663252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.724537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e528d5-b9fa-4edc-b43b-24066d5d8180 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.758711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486122-2758-471f-8346-28cd487cb44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.759337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486122-2758-471f-8346-28cd487cb44c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.759619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486122-2758-471f-8346-28cd487cb44c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.759891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486122-2758-471f-8346-28cd487cb44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.770281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c4c02-d204-4bdb-a5d8-7100f4db867a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.771299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486122-2758-471f-8346-28cd487cb44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.772762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486122-2758-471f-8346-28cd487cb44c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.781039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-23486122-2758-471f-8346-28cd487cb44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:33.793863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:33.795196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:33.796261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:33.796261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:33.796261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:33.796261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:33.798774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 466/940] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:33.798774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486122-2758-471f-8346-28cd487cb44c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.812550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.813177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.813491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.813771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.814990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:33.815380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.816973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.824729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 475/941] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:33.825547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.904348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:33.915997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486122-2758-471f-8346-28cd487cb44c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.928811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd30f92-7509-432c-a65d-3a53906e44c8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:33.952404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.953049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.953362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.953639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.961783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.963255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.965019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:33.965913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:33.966339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:33.966756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:33.967591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:33.968041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:33.969610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:33.976091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:33.977118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:33.978219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:33.981343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 467/942] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:33.982310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:33.988024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 476/943] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:33.988999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:34.023697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:34.098345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8d86a-412a-4141-9162-71717fa55c4c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:34.101409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17098-f007-483f-92b9-9fe15d4173c9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:34.133731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:34.134349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:34.134909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:34.134909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:34.135597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:34.135898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:34.137287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:34.137673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:34.137907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:34.138189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:34.138488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:34.139143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:34.139446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:34.140804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:34.145677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 468/944] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:34.146455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:34.157760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 477/945] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:34.158697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:34.174929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:34.276051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:34.281169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4ad38-d172-4996-bc82-b2873248640e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:34.289248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6330fa-da7a-444c-927b-65c091d3393e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:34.320190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:34.321223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:34.321647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:34.322060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:34.326140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:34.326955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:34.327312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:34.327660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:34.328464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:34.330649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:34.332310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:34.334056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:34.347706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:34.348628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:34.349936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 469/946] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:34.356095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:34.445307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:34.470146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8805f1-d48e-4373-b214-83d208e7f4c5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:34.508193 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:34.508860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:34.509146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:34.509447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:34.510195 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:34.510501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:34.512018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d9a70-d055-4b21-8cec-e0d09def0479 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:34.526591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 478/947] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:34.532057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 470/948] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:34.533401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:34.659991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a9f57-189f-46e0-a682-c3fdf8240873 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:34.699001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c28336-a14a-49c1-bc09-e7fab176945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:34.699725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c28336-a14a-49c1-bc09-e7fab176945d None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:34.700073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c28336-a14a-49c1-bc09-e7fab176945d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:34.700415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c28336-a14a-49c1-bc09-e7fab176945d None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/4444ac94fc184e949c285d365931d0f2/roles/841751d723ec454aae186b4da332a393` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:34.702866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c28336-a14a-49c1-bc09-e7fab176945d None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=4444ac94fc184e949c285d365931d0f2, role_id=841751d723ec454aae186b4da332a393)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:34.704373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c28336-a14a-49c1-bc09-e7fab176945d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:34.774664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 479/949] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:34 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/4444ac94fc184e949c285d365931d0f2/roles/841751d723ec454aae186b4da332a393 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:22:35.328247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b914ca-32df-4245-80d5-f2833e316132 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:35.328784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b914ca-32df-4245-80d5-f2833e316132 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:35.328784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b914ca-32df-4245-80d5-f2833e316132 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:35.331055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 471/950] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:35.336537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:35.337389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:35.337896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:35.345469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:35.358809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:35.360672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:35.437511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 480/951] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:35.460578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b914ca-32df-4245-80d5-f2833e316132 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:35.567040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b914ca-32df-4245-80d5-f2833e316132 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:35.601973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:35.602896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:35.603382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:35.603890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:35.604714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:35.606375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:35.608316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:35.618725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:35.625958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:35.627454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 472/952] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:35.630924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:35.736336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0971f-a07c-406b-aede-afcd3ddc2c04 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:35.774789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:35.775415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:35.775696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:35.775968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:35.776649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:35.777020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:35.778446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:35.779566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:35.784805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 481/953] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:35.791489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:35.900503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cdf5e-03ee-4fe5-82eb-3650c84d9ba8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:35.938165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:35.939248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:35.939885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:35.940668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:35.941627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:35.942142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:35.943728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7443391-aab0-4d35-86f4-ba47d2ccda6e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:35.953294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 473/954] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:35.956291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:36.064272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a4ccf-7c33-4a7f-bacd-21643c3b86fa None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:36.099260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebe91d-4ff1-4669-a81a-0fc11b89e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:36.099884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe91d-4ff1-4669-a81a-0fc11b89e3b8 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.100164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe91d-4ff1-4669-a81a-0fc11b89e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.100438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe91d-4ff1-4669-a81a-0fc11b89e3b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.101140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebe91d-4ff1-4669-a81a-0fc11b89e3b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:36.102536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebe91d-4ff1-4669-a81a-0fc11b89e3b8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:36.188214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 482/955] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:36.282395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79791fb-28af-4210-ba24-7f737147fb01 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.282752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79791fb-28af-4210-ba24-7f737147fb01 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.283067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79791fb-28af-4210-ba24-7f737147fb01 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.284924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 474/956] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:36.290651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.291307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.291821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.311906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:36.313626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:36.371699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:36.397821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 483/957] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:36.422941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79791fb-28af-4210-ba24-7f737147fb01 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:36.493459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:36.537520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79791fb-28af-4210-ba24-7f737147fb01 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:36.572950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:36.573607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.573877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.574145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.574833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:36.576266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:36.578813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING keystone.server.flask.common [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 03:22:36.618024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.sql.core [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 03:22:36.618024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 03:22:36.618024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [parameters: {'user_id': '141b8db821d747e1b66cc2fab5cdb9bb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 03:22:36.618024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80713) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-c1f54460-0864-4504-9c22-76cd129cc232 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application conn.query(q) Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application result.read() Aug 04 03:22:36.635542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 03:22:36.636548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application next(self.gen) Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application yield resource Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application next(self.gen) Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise self.value Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application session.commit() Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 03:22:36.637510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self.session.flush() Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application compat.raise_( Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 03:22:36.638374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 03:22:36.639248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application conn.query(q) Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application result.read() Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 03:22:36.640094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [parameters: {'user_id': '141b8db821d747e1b66cc2fab5cdb9bb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:36.640911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 03:22:36.641932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.643718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 475/958] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Aug 4 03:22:36 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:36.646967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:36.679341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd63a2-4eb9-48b2-89b7-bbb0fc617b21 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.679624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd63a2-4eb9-48b2-89b7-bbb0fc617b21 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.679970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd63a2-4eb9-48b2-89b7-bbb0fc617b21 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.684373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 476/959] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:36.690774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.691065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.691705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.713589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:36.715226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:36.756364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b398-2590-4d97-be0e-42acbc78ebe9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:36.797063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 477/960] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:36.800780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:36.801715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.802196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.802661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.810607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:36.812822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:36.814004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:36.830475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:36.831569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 03:22:36.832489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:36.836848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 484/961] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:36.838453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:36.889519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:36.893721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:36.940356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1cb0e3-aa87-4674-be9c-d8006955ef31 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:36.961594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadd46-1ab6-4b9b-88cb-4eb358bf58d5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:36.977054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:36.977728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.978013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.978282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.978919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:36.980435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ae5919fd-352f-4f53-ae76-8d9ad9920e25 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:36.990789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:36.991803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:36.992683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 478/962] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:36.993876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17dc0f-a68c-4f16-b4c9-438008a1382b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:36.993876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17dc0f-a68c-4f16-b4c9-438008a1382b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:36.993876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17dc0f-a68c-4f16-b4c9-438008a1382b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:36.997243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 479/963] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:36.997616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17dc0f-a68c-4f16-b4c9-438008a1382b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:37.013123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.013986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.014633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.014934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.015988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:37.016300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:37.019027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72747872-9b60-4e25-a43c-2ab8a1d6754d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:37.077824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 485/964] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Aug 4 03:22:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:37.079016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.079320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.079601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.099167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:37.100446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:37.119547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17dc0f-a68c-4f16-b4c9-438008a1382b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:37.148708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:37.174380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.175614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.176693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.177715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.178785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:37.179665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:37.181382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:37.193408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 486/965] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:37.193891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 480/966] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:37.194690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:37.222084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:37.233557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:37.323652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42baa-6fa9-453c-af9d-0c9a662a0dc8 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:37.354768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7eff7f-a8be-47b1-bb46-ac12697ed270 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:37.366436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.367324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.367671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.368013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.368732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:37.369315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:37.370532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:37.376135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 487/967] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:37.398143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.398862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.399082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.399444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.412317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:37.417858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:37.418850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:37.419678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 481/968] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Aug 4 03:22:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:37.425514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:37.530782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:37.543600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f575-be72-48ce-ab43-16ec0473ef29 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:37.578851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.579483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.579769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.580064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.580783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:37.581085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:37.582569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:37.596035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 488/969] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:22:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:37.597584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:37.641184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:37.700453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259ab6-0146-4dc6-825a-0c7b756a6ad9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:37.733971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.734621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.734900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.735271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.746351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:37.747809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:37.762752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:37.763667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:37.764574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:37.764574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:37.764574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:37.764574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:37.765344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 482/970] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:37.769961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:37.875593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52e230-1fbb-4add-b6be-e3601c0c8286 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:37.911481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:37.912107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:37.912389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:37.912666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:37.913399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:37.913707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:37.915097 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:37.931657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 489/971] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:37.932917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.028345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.035461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179c0d-d546-42b2-91b2-6e39f7b3e932 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.069746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.070720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.071020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.071297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.079068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.081228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:38.092263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:38.093167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:38.099016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 483/972] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:22:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:38.099612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.115552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.210808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e0fb4-df48-4365-b924-dc49a2ff1560 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.245604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.246227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.246517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.246794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.247583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:38.247892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.249364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88640d-b9a6-49e5-a8ce-464d80163d44 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.268788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 490/973] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:38.274432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.367696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927f00-aebc-4ca0-aac3-80268f13aa47 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.372640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927f00-aebc-4ca0-aac3-80268f13aa47 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.372640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea927f00-aebc-4ca0-aac3-80268f13aa47 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.372640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 491/974] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:38.374293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.374577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.374855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.377596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d352bb3-a81d-4f7e-b9ef-9a252be855ee None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.393662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:38.397601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:38.414033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.414652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.414938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.415215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.441821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.467786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 492/975] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:38.491066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.492282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.493320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.493830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.593452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f827a83f-69de-4409-aed3-9483b91028a1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.613362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 484/976] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:38.618977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.633245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.633891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.634172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.634445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.635070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.636444 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:38.645745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:38.646699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:38.647508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 493/977] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:38.652650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.683816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.721442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.729802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2f669-8787-4e47-bb88-2dd4766d4f4e None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.758206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f55fb6-4f24-4eaa-81cb-8c3f9d37ce60 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.768111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.769097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.769417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.769697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.780160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.781664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.795228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.795842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.796125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.796393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.797060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:38.797313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:38.798163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:38.804641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:38.804641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:38.804641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:38.804641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:38.804860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.804932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 485/978] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:38.804999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.804999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 494/979] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:38.809138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.809396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.911847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb45502-d145-42da-a390-4df5774f80ae None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.917470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1cf6f-ca0d-48b9-9ded-727a3d50429c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:38.918655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.947169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.947835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.948186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.948293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:38.949243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.949952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:38.950493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.952030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.954872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:38.955504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:38.958239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:38.958239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:38.958239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:38.958239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:38.958515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:38.968011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 495/980] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:38.968165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 486/981] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:38.970140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:38.970247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:39.077890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87c749-27c1-43fe-b78f-b9e96b38cd2b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:39.078788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a6efd-7b13-42a5-a5ba-020bda43785c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:39.114815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:39.115433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:39.115571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.115754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.115976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.116080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.116318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.116592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.117385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] RBAC: Adding query filter params (interface=admin, service_id=f97517b623c048d1a864d94424112e99, region_id=RegionOne) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:39.117696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:39.119078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:39.123229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:39.124539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 496/982] 10.222.0.66 () {56 vars in 1249 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/endpoints?service_id=f97517b623c048d1a864d94424112e99&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:39.124786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:39.130830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:39.137089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:39.138160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:39.139254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:39.145249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 487/983] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:39.145831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:39.176031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:39.206285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:39.248847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51c2cb-396f-4b0b-bd47-5906fab3ada6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:39.255760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcccdbc-7c60-403a-8d26-919846aafb93 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:39.290974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:39.291589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.291870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.292080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:39.292192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.292705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.292863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:39.292972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.293166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:39.293294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.293918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:39.294617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931b7d9-1a55-4dad-a7aa-e8db6774d1be None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:39.295278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:39.301860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 497/984] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:39 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:22:39.307244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 488/985] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:39.313383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:39.417914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1184c-cf20-41e3-9465-a0b7e0f147ce None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.418204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1184c-cf20-41e3-9465-a0b7e0f147ce None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.418484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1184c-cf20-41e3-9465-a0b7e0f147ce None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.420047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 489/986] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:39.425975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5675-f441-482b-aded-64db3e692e7a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.426200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5675-f441-482b-aded-64db3e692e7a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.426461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b5675-f441-482b-aded-64db3e692e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.433079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd862b53-1ee1-406a-852d-f3cd26e26a83 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:39.447267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-221b5675-f441-482b-aded-64db3e692e7a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:39.448487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-221b5675-f441-482b-aded-64db3e692e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:39.459892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:39.473607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:39.475352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.475352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.475352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.475595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] RBAC: Adding query filter params (role.id=841751d723ec454aae186b4da332a393, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=4444ac94fc184e949c285d365931d0f2) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:39.475808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:39.477342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:39.484393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 498/987] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:39 2022] GET /identity/v3/role_assignments?user.id=4444ac94fc184e949c285d365931d0f2&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=841751d723ec454aae186b4da332a393 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:39.529574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 490/988] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:39.552252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:39.675718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3131-0092-4ff6-8cff-a8b811474303 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:39.712662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c918d7d-c168-407f-82cf-4ecdf7658e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:39.713383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918d7d-c168-407f-82cf-4ecdf7658e28 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:39.713766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918d7d-c168-407f-82cf-4ecdf7658e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:39.714140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c918d7d-c168-407f-82cf-4ecdf7658e28 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:39.714930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c918d7d-c168-407f-82cf-4ecdf7658e28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:39.716399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c918d7d-c168-407f-82cf-4ecdf7658e28 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:39.772987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 499/989] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:39 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:41.378374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036203-7609-4c5b-b4e8-e9c74b4ab71b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.378929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036203-7609-4c5b-b4e8-e9c74b4ab71b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.378929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036203-7609-4c5b-b4e8-e9c74b4ab71b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.380445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 491/990] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:41.385774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b72884-28e0-493f-afc9-3adebfc4aa02 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.386169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b72884-28e0-493f-afc9-3adebfc4aa02 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.386337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b72884-28e0-493f-afc9-3adebfc4aa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.405039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-53b72884-28e0-493f-afc9-3adebfc4aa02 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:41.406351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-53b72884-28e0-493f-afc9-3adebfc4aa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:41.450468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:41.485232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 500/991] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:41.505870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036203-7609-4c5b-b4e8-e9c74b4ab71b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:41.524791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:41.584797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca3b0d-2134-421f-89f4-1e4a6a4a55ed None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.585122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca3b0d-2134-421f-89f4-1e4a6a4a55ed None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.585408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca3b0d-2134-421f-89f4-1e4a6a4a55ed None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.586934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 501/992] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:41.591607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.591894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.592171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.610672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:41.612596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:41.614319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036203-7609-4c5b-b4e8-e9c74b4ab71b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:41.657312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:41.658195 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.658315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.658595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.659233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:41.660617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:41.670247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:41.671295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:41.672164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 492/993] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Aug 4 03:22:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:41.678127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:41.685242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 502/994] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:41.707840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:41.768490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:41.791331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81e1b-fef5-4f39-b31d-98fdd769f646 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:41.813904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a197a-63ff-4f1a-90b8-977a2ea3f414 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:41.815874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:41.832163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:41.832784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.833061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.833360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.834086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:41.834386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:41.835759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e8d5f-bbc1-4cdc-8c38-04f6c075f08c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:41.840638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 493/995] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:41.841826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a08ff-f41e-4476-9dd3-092b6264c73f None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.842115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a08ff-f41e-4476-9dd3-092b6264c73f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.842389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a08ff-f41e-4476-9dd3-092b6264c73f None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.843759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 494/996] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:41.848828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a08ff-f41e-4476-9dd3-092b6264c73f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:41.850253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:41.850897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.851177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.851455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] PATH_INFO: `/v3/services/image` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.852083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:41.853502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-4cfbe710-a38e-4104-8aa4-e83bd9f9c6fd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:41.862801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:41.863764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:41.864797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 503/997] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Aug 4 03:22:41 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:41.866280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.866573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.866853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.884351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:41.885579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:41.951373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a08ff-f41e-4476-9dd3-092b6264c73f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:41.951994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 504/998] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:41.952873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:41.987713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:41.988250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:41.988534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:41.988809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:41.989511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:41.989820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:41.991207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.000734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 495/999] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Aug 4 03:22:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:42.000734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.008369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:42.049074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:42.055190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b362e70-2afd-4dad-89f1-bb499b8aced7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.089397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.090027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.090322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.090643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.091329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] RBAC: Adding query filter params (name=image) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:42.091632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.093018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.097541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843ebae-5329-4cb3-8683-b14802f48d12 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.098508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 505/1000] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:42.099101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.131419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.132058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.132336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.132614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.133263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.134634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:42.143818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:42.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:42.145913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 496/1001] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:42.146687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.199722 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e9b5e-1c84-440a-b631-aa39bec58020 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.219693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:42.233150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.233818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.234101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.234371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.235076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.237052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.250779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e844b-9403-46f5-ad78-8371ae8a299c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.285082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.285721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.285998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.286270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.286518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 506/1002] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:42.286944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:42.287247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.287344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.288623 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.293622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 497/1003] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:42.325135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:42.431358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4410a1-170d-4bf2-8593-35e6752ed967 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.473684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.474367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.474655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.474933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.475665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:42.475976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.477423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.483896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 507/1004] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:42.490678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.597760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb992c87-a9b7-4cf0-9888-a7100d7203a8 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.632757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.633420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.633706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.633979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.634660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:42.634964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.636341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.638567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:42.642572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 498/1005] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:42.648736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.731145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:42.751010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf4e48-a67d-44bc-b131-78f9ca96f712 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.787647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.788279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.788564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.788898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.789685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] RBAC: Adding query filter params (interface=public, service_id=6ffee5c41801494b8edff6d662ec2efd, region_id=RegionOne) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:42.790000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.791379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fadae9-c1a4-4ea1-90cb-f1c9ed5f20db None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.796898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 508/1006] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:42 2022] GET /identity/v3/endpoints?service_id=6ffee5c41801494b8edff6d662ec2efd&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:42.802859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:42.904877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8afca-b0a9-4d17-9420-ba98252c92b2 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:42.938765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70881e18-08e3-47f5-af7c-865439616305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:42.939415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70881e18-08e3-47f5-af7c-865439616305 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:42.939769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70881e18-08e3-47f5-af7c-865439616305 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:42.939974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70881e18-08e3-47f5-af7c-865439616305 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:42.940621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70881e18-08e3-47f5-af7c-865439616305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:42.942037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70881e18-08e3-47f5-af7c-865439616305 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:42.948566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 499/1007] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:42 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:22:44.311789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104848-6a72-4ec2-aa73-2ea1b4eb07d4 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.312328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104848-6a72-4ec2-aa73-2ea1b4eb07d4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.312328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d104848-6a72-4ec2-aa73-2ea1b4eb07d4 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.313843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 509/1008] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:44.319034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.319445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.319603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.337599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd663d45-81f0-41d4-8edc-e35d83176337 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.337900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd663d45-81f0-41d4-8edc-e35d83176337 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.337999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:44.338150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd663d45-81f0-41d4-8edc-e35d83176337 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.339155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:44.339620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 510/1009] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:44.344229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.344517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.344798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.363608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:44.365057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:44.412009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:44.425312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 500/1010] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:44.437247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 511/1011] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:44.445811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:44.459832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:44.537039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:44.554048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2776-e836-48ca-96ea-b98abd085044 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:44.562923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c075d4-31dd-482d-8d3e-d3c258bfac9b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:44.596695 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40980c6b-0319-4480-abd0-ca109db15e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:44.597703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40980c6b-0319-4480-abd0-ca109db15e83 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.598182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40980c6b-0319-4480-abd0-ca109db15e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.598666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:44.598888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40980c6b-0319-4480-abd0-ca109db15e83 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.599305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.599584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.599856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.600409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40980c6b-0319-4480-abd0-ca109db15e83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:44.600999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:44.602214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40980c6b-0319-4480-abd0-ca109db15e83 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:44.602470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:44.604074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 512/1012] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:44.609661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:44.706145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 501/1013] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:44 2022] POST /identity/v3/services => generated 242 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:44.715745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1c4fc-dc35-4de2-93c7-1e2f0cb4abbb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:44.763409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33b892-b4bc-41e1-a554-37c1d64c5ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:44.764045 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b892-b4bc-41e1-a554-37c1d64c5ec1 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.764320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b892-b4bc-41e1-a554-37c1d64c5ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.764586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33b892-b4bc-41e1-a554-37c1d64c5ec1 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.765324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b892-b4bc-41e1-a554-37c1d64c5ec1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:44.766821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33b892-b4bc-41e1-a554-37c1d64c5ec1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:44.778044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:44.872317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 513/1014] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:22:44 2022] POST /identity/v3/projects => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:44.976824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7dc8c-ad8f-4d4c-8224-cdd20552e734 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.977013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7dc8c-ad8f-4d4c-8224-cdd20552e734 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.977329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7dc8c-ad8f-4d4c-8224-cdd20552e734 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:44.978867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 502/1015] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:44.984480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:44.984774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:44.985043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:45.004833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:45.006101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:45.078597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 514/1016] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:45.101791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7dc8c-ad8f-4d4c-8224-cdd20552e734 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:45.167053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:45.216879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7dc8c-ad8f-4d4c-8224-cdd20552e734 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:45.252361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:45.253183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:45.253493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:45.253765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:45.254402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:45.255782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:45.264665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:45.265693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 03:22:45.266639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:45.268622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 503/1017] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:45.272793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:45.375663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf84f2-be3d-45df-9573-8a4f8a5207e9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:45.396995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:45.411430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:45.412056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:45.412338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:45.412613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:45.413313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:45.413616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:45.414996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:45.419960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 515/1018] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:45.426429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:45.530237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7541d6-7c98-47c1-afc8-9cea9c52e32b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:45.548479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:45.565723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:45.566343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:45.566636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:45.566898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:45.567581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] RBAC: Adding query filter params (type=compute) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:45.567886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:45.569316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd473fc1-688e-4dc6-be35-0279cd3174ac None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:45.579055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 504/1019] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Aug 4 03:22:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:45.580825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:45.684027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3840ec3-aa9d-4728-ad6a-afeb4e361660 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:45.718344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be77954b-a171-4d63-b525-1fa50b73fe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:45.719240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77954b-a171-4d63-b525-1fa50b73fe9f None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:45.719706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77954b-a171-4d63-b525-1fa50b73fe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:45.720696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be77954b-a171-4d63-b525-1fa50b73fe9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:45.721696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77954b-a171-4d63-b525-1fa50b73fe9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:45.723281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be77954b-a171-4d63-b525-1fa50b73fe9f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:45.784024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 516/1020] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:46.766280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba012a-ea39-4888-a3c7-24e5b362f372 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:46.766801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba012a-ea39-4888-a3c7-24e5b362f372 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:46.766879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba012a-ea39-4888-a3c7-24e5b362f372 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:46.768838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 505/1021] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:46.773781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c621664-e0eb-4df5-873e-71e7b4af5579 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:46.774185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c621664-e0eb-4df5-873e-71e7b4af5579 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:46.774366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c621664-e0eb-4df5-873e-71e7b4af5579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:46.793567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0c621664-e0eb-4df5-873e-71e7b4af5579 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:46.794748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0c621664-e0eb-4df5-873e-71e7b4af5579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:46.830404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:46.873530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 517/1022] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:46.891901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba012a-ea39-4888-a3c7-24e5b362f372 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:46.922891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950459e-e265-4d2b-97b0-d0a2c1563e49 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:46.923184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950459e-e265-4d2b-97b0-d0a2c1563e49 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:46.924012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950459e-e265-4d2b-97b0-d0a2c1563e49 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:46.925391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 518/1023] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:46.930171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2657153-2832-4473-a3a0-0785071d533d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:46.930450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2657153-2832-4473-a3a0-0785071d533d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:46.930720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2657153-2832-4473-a3a0-0785071d533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:46.949251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e2657153-2832-4473-a3a0-0785071d533d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:46.950427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e2657153-2832-4473-a3a0-0785071d533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:47.004560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba012a-ea39-4888-a3c7-24e5b362f372 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.025324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 519/1024] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:47.044026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.044933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.045426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.045904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] PATH_INFO: `/v3/services/image` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.046719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.048557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.049495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2657153-2832-4473-a3a0-0785071d533d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:47.058294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:47.059260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:47.060072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 506/1025] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Aug 4 03:22:46 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:47.065839 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:47.116669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:47.150743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:47.157620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2657153-2832-4473-a3a0-0785071d533d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.184281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086a9e-e24e-41f9-90d9-b9b579637185 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.198919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.199774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.199910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.200099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.201319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.202702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.204443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 520/1026] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:47.210658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:47.221264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.221889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.222167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.222437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.223103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] RBAC: Adding query filter params (name=image) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:47.223408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.224768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.230376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 507/1027] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:47.235583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:47.319268 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5b98-7df2-4bfc-afbf-2e511c8889a9 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.350054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bfd1d-ab71-4c84-96cf-6eee4584f745 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.355192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.355811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.356090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.356365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.357039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.358578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.362328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:47.387794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.388894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.389441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.389984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.390666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] RBAC: Adding query filter params (type=image) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:47.390971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.392363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.397418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 508/1028] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:47.403178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:47.493846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 521/1029] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:22:47 2022] POST /identity/v3/users => generated 285 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:22:47.507892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36080c9-6bef-453b-a174-a79a0ec9bb8f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.545967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.546613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.546893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.547164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.547950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] RBAC: Adding query filter params (interface=public, service_id=08c9e7dcaf3f43f2bcb848410a5b3bfc, region_id=RegionOne) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:47.548783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.550536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca037b-cc8b-4cfa-9956-b93a9e6a417c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.561118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 509/1030] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:22:47 2022] GET /identity/v3/endpoints?service_id=08c9e7dcaf3f43f2bcb848410a5b3bfc&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:47.562196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:47.587525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:47.672093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ab3d4-53cb-4f82-91bc-f5627fa14e78 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:47.709497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f94d-bd72-4628-a363-c2bbc9b5b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:47.710391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f94d-bd72-4628-a363-c2bbc9b5b2ca None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.711310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f94d-bd72-4628-a363-c2bbc9b5b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.711310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f94d-bd72-4628-a363-c2bbc9b5b2ca None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.711577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8f94d-bd72-4628-a363-c2bbc9b5b2ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:47.713850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8f94d-bd72-4628-a363-c2bbc9b5b2ca None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:47.719990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 522/1031] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:22:47 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:22:47.932864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d247a-9f72-4f3f-9430-8781be592c08 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.933449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d247a-9f72-4f3f-9430-8781be592c08 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.933449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d247a-9f72-4f3f-9430-8781be592c08 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.934931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 510/1032] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:47.940667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:47.941048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:47.941251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:47.960192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:47.961548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:48.031794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 523/1033] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:48.055099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d247a-9f72-4f3f-9430-8781be592c08 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:48.098655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:48.163406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d247a-9f72-4f3f-9430-8781be592c08 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:48.201626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:48.202192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:48.202472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:48.202743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:48.213089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:48.217979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:48.218870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:48.225252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 511/1034] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Aug 4 03:22:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:48.225968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:48.241032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:48.326905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd1ca1-7e27-42b2-96aa-cfa925070cc2 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:48.360539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:48.361719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:48.362038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:48.362275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:48.362992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:48.363297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:48.364680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:48.377341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 524/1035] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:22:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:48.380108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:48.478593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:48.482966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd549df-6ceb-4d46-8b85-0ade9c755720 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:48.517128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:48.517957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:48.518239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:48.518514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:48.528996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:48.530483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:48.544954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:48.545899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:48.546780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:48.546780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:48.546780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:48.546780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:48.546780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 512/1036] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:48.553566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:48.634753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:48.656987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a014-506b-47ef-a0ed-c2ef8732b99f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:48.694394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:48.695014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:48.695293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:48.695563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:48.696246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:48.696544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:48.697937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:48.711885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 525/1037] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:48.716261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:48.823659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7900d7a-25af-494e-a566-4636a9ea8672 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:48.860612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:48.862224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:48.862512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:48.862787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:48.870612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:48.872055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:48.883510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:48.884427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:48.885773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 513/1038] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:48.890436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:48.993598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4858f8d-c59b-4aa3-b641-6fccaf7e6ebf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:49.027947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:49.028567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.028848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.029122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.029948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:49.030256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:49.031692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:49.034883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.054676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 526/1039] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:49.055813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.068794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.158637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc2411-9a4b-4824-919e-d7f52055fc80 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:49.195803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:49.196429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.196714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.197053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.267033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:49.268661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:49.280055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.375258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 514/1040] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 03:22:49.377587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.481999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d4b4-9a8c-43d4-83ea-412d8d7965ba None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:49.516763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:49.517978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.518257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.518524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.528785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:49.530387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-bcc048cf-cdc6-4a14-8453-4d8366cd2f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:49.544862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:49.545943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:49.551583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:49.551583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:49.551583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:49.551583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:49.551583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 527/1041] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:49.553263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.581192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4d314-a03c-46c7-915d-4ed9714e8b07 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.581512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4d314-a03c-46c7-915d-4ed9714e8b07 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.581775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4d314-a03c-46c7-915d-4ed9714e8b07 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.583224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 528/1042] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:49.588987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.589347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.589637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.609015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:49.610272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:49.655631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.667338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72bcce-0bbd-4e3e-9426-0ab1c7d1cc0c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:49.681168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 529/1043] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:49.683355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.704009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.707200 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:49.708729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.709008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.709312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.710017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:49.710324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:49.711700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:49.724957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 515/1044] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:49.731082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.813799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c595c-540a-40a8-9c60-cd9c97eaeffd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:49.830742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7eead-c74b-41ab-8b6e-84cb1ba81272 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:49.849964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f69abb-2f56-4195-8ade-73415a26424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:49.850606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f69abb-2f56-4195-8ade-73415a26424b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.850888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f69abb-2f56-4195-8ade-73415a26424b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.851156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f69abb-2f56-4195-8ade-73415a26424b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.862582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f69abb-2f56-4195-8ade-73415a26424b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:49.865721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:49.866387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.866689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.866944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-92f69abb-2f56-4195-8ade-73415a26424b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:49.867422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:49.868377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:49.869282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 530/1045] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:49.870749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773465c-af79-4a46-a702-9e590e707819 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.870749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773465c-af79-4a46-a702-9e590e707819 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.870749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773465c-af79-4a46-a702-9e590e707819 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.871487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 531/1046] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:49.874772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773465c-af79-4a46-a702-9e590e707819 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.878644 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:49.880122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-432fd0f9-cb1a-4632-8246-83cadb15b437 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:49.891645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:49.893285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:49.894165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 516/1047] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:49.894662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:49.894729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:49.894989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:49.912731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:49.913938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:49.920354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.936761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:49.978960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 517/1048] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:49.979780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:49.984752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773465c-af79-4a46-a702-9e590e707819 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.020663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.021311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.021587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.021864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.022584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:50.022888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.024263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.032360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 532/1049] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:50.033180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.085118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbcf82-34ab-42f8-bfa3-aa1e7ae761da None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.117781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.118384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.118660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.118930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.119670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:50.119970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.121424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.128628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.132449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d86ef-1f2e-4f64-b42f-a2df6f4fd207 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.133397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 518/1050] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:50.137230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.159315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.167451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.168097 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.168382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.168836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.169349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.170739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:50.180916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:50.181940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:50.182790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 533/1051] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:50.183552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.240912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ace5f3-a4bc-4d25-a207-ffc4a2be412c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.274585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.275216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.275493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.275761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] PATH_INFO: `/v3/users/0a52f4a328a04a5981bb34b17a16385f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.286287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf274099-aaf3-4a01-8f26-a0c25ebedac7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.289034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] RBAC: Authorizing `identity:get_user(user_id=0a52f4a328a04a5981bb34b17a16385f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.290505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.292131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 519/1052] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/users/0a52f4a328a04a5981bb34b17a16385f => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:50.292878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.320506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.321418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.321703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.321979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.322780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] RBAC: Adding query filter params (role.id=841751d723ec454aae186b4da332a393, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=09fbb54e522c47d89de166682742336f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:50.323086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.324528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.330164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 534/1053] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/role_assignments?user.id=09fbb54e522c47d89de166682742336f&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=841751d723ec454aae186b4da332a393 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:50.330944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.368091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.390207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.414282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ffdd9-4cb5-4d67-9b84-9f120930cd11 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.451928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.452658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.453011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.453378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.454145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:50.455082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.455699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307590-6327-4d87-9c38-6c2f8fb58929 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.456625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.467556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 520/1054] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:50.468672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.500133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.500757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.501035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.501393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.574935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.576550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.584099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5552c4-f9dc-40c9-a208-eff2c8ad7fd7 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.604496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.620049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.620596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.620878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.621159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.622989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] RBAC: Adding query filter params (type=image) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:50.623322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.624751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.635539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 521/1055] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:50.637249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.646956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.687567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 535/1056] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:50.689356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.751949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b072c8ef-d636-43af-a2a2-fc97a6f99b76 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.788647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ea42-56e6-4f8f-a3a2-e1e91bec5248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.789458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ea42-56e6-4f8f-a3a2-e1e91bec5248 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.789840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ea42-56e6-4f8f-a3a2-e1e91bec5248 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.790212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ea42-56e6-4f8f-a3a2-e1e91bec5248 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.791061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08ea42-56e6-4f8f-a3a2-e1e91bec5248 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.792540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08ea42-56e6-4f8f-a3a2-e1e91bec5248 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.797036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bb39c-b0e1-410a-a18e-a40c476b867d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.824926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:50.834368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.835039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.835323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:50.835599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:50.836762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] RBAC: Authorizing `identity:get_project(project_id=50f0037c482d4504a1db39b4a2995bd9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:50.838239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:50.840169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 536/1057] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:50.845937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:50.880997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 522/1058] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:22:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:22:50.956507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caf3f7-4608-4cfd-8dd4-ee3801cc0abf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:50.998938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:50.999570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:50.999854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:51.000131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:51.000938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=50f0037c482d4504a1db39b4a2995bd9, user.id=0a52f4a328a04a5981bb34b17a16385f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:51.001284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:51.003573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c11fb-992f-4527-855c-196b1b70e985 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:51.010025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 537/1059] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:50 2022] GET /identity/v3/role_assignments?user.id=0a52f4a328a04a5981bb34b17a16385f&scope.project.id=50f0037c482d4504a1db39b4a2995bd9&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:52.338365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ae69-0564-4a50-98b7-900cf0e39cc7 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.338898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ae69-0564-4a50-98b7-900cf0e39cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.338898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ae69-0564-4a50-98b7-900cf0e39cc7 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.340442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 523/1060] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:52.345790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.346206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.346380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.364651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:52.365862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:52.416824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:52.440182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 538/1061] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:52.458853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39ae69-0564-4a50-98b7-900cf0e39cc7 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:52.523000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:52.563740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39ae69-0564-4a50-98b7-900cf0e39cc7 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:52.598441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:52.599084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.599361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.599629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.611513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:52.616416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:52.617556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:52.623552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 524/1062] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Aug 4 03:22:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:52.624258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:52.727662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c8b2-7a07-4922-862c-4fc0a2837e55 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:52.761093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:52.761737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.762018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.762287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.763000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:52.763299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:52.764672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548b7a-29a9-4782-96dd-bc6627f93722 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:52.777246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 539/1063] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:22:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:52.779967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:52.882917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968b38-bcc1-476a-a978-a282500e955f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:52.906067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:52.918257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:52.918898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.919203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.919447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.919700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f07890-7bae-4d02-b253-b2b2a16a199d None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.920093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f07890-7bae-4d02-b253-b2b2a16a199d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.920452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f07890-7bae-4d02-b253-b2b2a16a199d None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.922373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 540/1064] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:52.927595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b1d5c-9b59-47e8-af1f-ebeb13d54542 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:52.927880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b1d5c-9b59-47e8-af1f-ebeb13d54542 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:52.928155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b1d5c-9b59-47e8-af1f-ebeb13d54542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:52.930637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:52.932088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:52.947251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:52.948152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:52.949015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:52.949015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:52.949015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:52.949015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:52.950369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-eb3b1d5c-9b59-47e8-af1f-ebeb13d54542 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:52.950703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 525/1065] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:52.951572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-eb3b1d5c-9b59-47e8-af1f-ebeb13d54542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:52.954344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:52.998138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.024317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 541/1066] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:53.025603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f42d9-39b9-4c12-b0f7-38b26c8e07a0 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.025891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f42d9-39b9-4c12-b0f7-38b26c8e07a0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.026164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f42d9-39b9-4c12-b0f7-38b26c8e07a0 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.029319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 542/1067] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:53.032541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.032945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.033250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.051431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:53.052605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:53.069527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547282-da5d-4f13-b586-fa269d1ede0a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.103069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.103789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.104079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.104348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.105036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:53.105368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.106741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.118827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 526/1068] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:53.119657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.125967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 543/1069] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:53.127304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.155732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.223172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d688d-a62e-4f24-b801-3166aed8b097 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.237364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc916b-6962-4539-b20d-8cf5200baacd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.244422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.257777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.258329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.258609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.258879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.272096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.272727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.273006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.273305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.280375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.281857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:53.293783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:53.294889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 544/1070] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:53.296095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.328886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.330960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.364322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.396793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e990ab-d686-41dd-a59a-3476a01e3c27 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.430642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.431272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 527/1071] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:53.431420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.431587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.431837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.432093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.442331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:53.447605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:53.448510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:53.449372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 545/1072] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:53.450083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.478336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.537030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73936f48-8954-489e-bf4e-0857ab58478c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.550499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3d8c4-e79a-4fac-a089-b9b38164c14c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.570978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.571595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.571878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.572152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.572946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:53.573282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.574727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.576418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.585442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.586080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.586361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.586662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.597758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 528/1073] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:53.597758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.601641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.603107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:53.618531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:53.619443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:53.620483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:53.620483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:53.620483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:53.620483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:53.620483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 546/1074] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:53.620483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.695964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535db08e-6a11-4745-8a15-912a55d6672b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.700208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.722482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89569b4-a95b-46e4-8f34-d3d5cdafa0cb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.729883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.730498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.730780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.731054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.731771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:53.732074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.733482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.741766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 529/1075] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:53.742682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.757309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.757946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.758220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.758493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.769171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.770647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:53.785582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:53.786641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:53.787592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:53.787592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:53.787592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:53.787592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:53.787592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 547/1076] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:53.787887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.804355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.848528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472a356-b0f3-4d99-a183-f3ba0a0d5ce9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.886447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.887064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.887342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.887613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.888308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:53.888612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.890025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.891820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e37af2-d8b9-4ba5-86cd-38922a316b73 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:53.901887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 530/1077] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:53.902681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:53.925761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:53.932529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:53.933225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:53.933526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:53.933804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] PATH_INFO: `/v3/users/0a52f4a328a04a5981bb34b17a16385f` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:53.935217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] RBAC: Authorizing `identity:get_user(user_id=0a52f4a328a04a5981bb34b17a16385f)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:53.936707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:53.943945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 548/1078] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/users/0a52f4a328a04a5981bb34b17a16385f => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:53.944974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:54.023182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760d746-866f-495e-accf-80c92b5ea243 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:54.040707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:54.062212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:54.063051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0abd43-8dee-4852-877a-9c907195a08c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:54.063187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:54.063440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:54.064265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:54.065347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:54.066243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:54.067641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:54.080124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 531/1079] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:54.086112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:54.107574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:54.108880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:54.108880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:54.109049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:54.111809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] RBAC: Authorizing `identity:get_project(project_id=50f0037c482d4504a1db39b4a2995bd9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:54.113533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:54.115437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 549/1080] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:22:53 2022] GET /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:54.121295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:54.206516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba24893-372f-46da-bf69-08fc07106b02 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:54.207492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:54.240882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:54.241555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:54.241840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:54.242116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:54.242614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8716ef-8858-4f84-8fed-5bb47ba5cba6 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:54.248787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:54.250281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:54.262429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:54.263449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:54.264511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 532/1081] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:54.270051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:54.279492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:54.280160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:54.280443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:54.280720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9/users/0a52f4a328a04a5981bb34b17a16385f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:54.283374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] RBAC: Authorizing `identity:create_grant(project_id=50f0037c482d4504a1db39b4a2995bd9, user_id=0a52f4a328a04a5981bb34b17a16385f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:54.284816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:54.303602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:54.349791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 550/1082] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:54 2022] PUT /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9/users/0a52f4a328a04a5981bb34b17a16385f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:22:54.369167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7878488-aa13-4d02-a1be-6da068d14782 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:54.406285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:54.406920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:54.407203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:54.407480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:54.408382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:54.409190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:54.410767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6893a-edd0-4031-93df-a9e3d14a6a48 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:54.424006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 533/1083] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:54.430161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:54.537328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988f103-dca1-405d-9b4b-aba07af4c248 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:54.561965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:54.571106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb54280-8c7a-4c83-9d8d-5b50900b88ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:54.571804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54280-8c7a-4c83-9d8d-5b50900b88ce None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:54.572060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54280-8c7a-4c83-9d8d-5b50900b88ce None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:54.572333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54280-8c7a-4c83-9d8d-5b50900b88ce None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/09fbb54e522c47d89de166682742336f/roles/841751d723ec454aae186b4da332a393` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:54.574548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54280-8c7a-4c83-9d8d-5b50900b88ce None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=09fbb54e522c47d89de166682742336f, role_id=841751d723ec454aae186b4da332a393)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:54.575997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54280-8c7a-4c83-9d8d-5b50900b88ce None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:54.640482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 551/1084] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:22:54 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/09fbb54e522c47d89de166682742336f/roles/841751d723ec454aae186b4da332a393 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:22:56.332731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b92e0-dd80-4640-a029-c515a0a3cf82 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.333288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b92e0-dd80-4640-a029-c515a0a3cf82 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.333288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b92e0-dd80-4640-a029-c515a0a3cf82 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.334645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 534/1085] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:56.339798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.340191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.340418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.358258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:56.359422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:56.427461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 552/1086] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:56.450184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b92e0-dd80-4640-a029-c515a0a3cf82 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:56.542526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:56.550499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b92e0-dd80-4640-a029-c515a0a3cf82 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:56.584802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505db04-35f8-4d0b-bd1b-92fcdb49f791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:56.585469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505db04-35f8-4d0b-bd1b-92fcdb49f791 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.585748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505db04-35f8-4d0b-bd1b-92fcdb49f791 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.586019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505db04-35f8-4d0b-bd1b-92fcdb49f791 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.596073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7505db04-35f8-4d0b-bd1b-92fcdb49f791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7505db04-35f8-4d0b-bd1b-92fcdb49f791 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:56.600851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:56.601836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:56.602690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 535/1087] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:22:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:56.608110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:56.631228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58216c-ea4b-48aa-b469-26463dfabc11 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.631510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58216c-ea4b-48aa-b469-26463dfabc11 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.631789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58216c-ea4b-48aa-b469-26463dfabc11 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.633289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 536/1088] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:22:56.637903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.638188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.638464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.656999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:56.658859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:56.708970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b3809-d43a-4c07-9dec-db041609ba61 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:56.725434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:56.726391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 537/1089] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:22:56.742698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:56.743529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.744266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.744527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.745269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:56.745573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:56.746948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:56.749425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:56.760860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 553/1090] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:22:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:56.761915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:56.832907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:56.848358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a72fb-e5db-41fe-8c7b-566c5bceb4e2 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:56.861456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4bd76-808d-4fb3-b94c-c7931dfe9f37 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:56.883477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:56.884112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.884389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.884670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.895558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:56.895788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:22:56.896192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:56.896476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:56.896752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] PATH_INFO: `/v3/users/0a52f4a328a04a5981bb34b17a16385f` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:56.897950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] RBAC: Authorizing `identity:get_user(user_id=0a52f4a328a04a5981bb34b17a16385f)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:56.899398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:22:56.900571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:56.901610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 03:22:56.902449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:56.903091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 554/1091] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:22:56 2022] GET /identity/v3/users/0a52f4a328a04a5981bb34b17a16385f => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:56.903160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 538/1092] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Aug 4 03:22:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:56.906632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:56.908023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:56.952761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.006404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a138b0b8-2037-42f5-a119-8c5efb48b2b1 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.007484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bdf15-aa46-437b-8a2c-420f5c8a11e2 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.040042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.040660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.040944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.041248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.041466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.042096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.042372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.042635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.043340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:57.043747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.045117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.053407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 539/1093] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:22:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:57.059008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.068021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.109090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.111100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.156103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.159116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead6e85-07a5-4144-9fec-b0ae9a391cad None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.194140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.194774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.195048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.195319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.206629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.208099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.213684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 555/1094] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:57.219125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:57.226944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:57.228000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:57.229065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:57.229065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:57.229065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:57.229065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:57.229065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 540/1095] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:57.233864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.292150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.320291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992579d3-bfcc-4d11-9a5a-e14e4e825e52 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.335007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d19b1-1115-4187-b359-a4d2868a8a7f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.354924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.355558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.355837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.356105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.357265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] RBAC: Authorizing `identity:get_project(project_id=50f0037c482d4504a1db39b4a2995bd9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.358699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.360342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 556/1096] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:57.366541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.369093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.369739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.370015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.370280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.370969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:57.371264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.372664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.382215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.385238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 541/1097] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:57.390766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.470592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b362ac-5cea-44e7-876e-4f65dd74e74a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.492171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d380527-c493-4bcd-bf54-456c13ee964a None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.504683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.505327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.505609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.505880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.506696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=50f0037c482d4504a1db39b4a2995bd9, user.id=0a52f4a328a04a5981bb34b17a16385f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:57.507001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.508436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.510009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.514157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 557/1098] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/role_assignments?user.id=0a52f4a328a04a5981bb34b17a16385f&scope.project.id=50f0037c482d4504a1db39b4a2995bd9&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:57.527549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.528220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.528503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.528778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.537297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.539114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:22:57.552823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:57.553900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:22:57.554849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 542/1099] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:57.560265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.645588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.667410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed505d-fec8-4913-843a-ad62dd57b3f4 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.702389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.703011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.703294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.703572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.704921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:57.705288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.706702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:57.723967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 558/1100] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:57.729854 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:57.827365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426f12f-f5f9-4523-b675-cc56d2f453b3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:57.860197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:57.861365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:57.861643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:57.861912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:57.912041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:57.933279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:57.934748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:58.031538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 543/1101] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:22:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:22:58.037128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:58.135422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aaa7d-8ea6-40d9-ac2c-6a2bb8d9ca81 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:58.171146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:58.171818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:58.172103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:58.172382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:58.175761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:58.183457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:58.185016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:22:58.199801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:58.200669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:58.201560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:22:58.201560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:22:58.201560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:22:58.201560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:58.201560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 559/1102] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:22:58.207358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:58.285751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:58.306021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ae239-068e-4181-b735-970384c02baf None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:58.339534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:58.340137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:58.340409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:58.340684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:58.341402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:58.341705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:58.343065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:58.353947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 544/1103] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:58.359637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:58.459004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7654c-7e68-4668-844e-692e29f0c52b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:58.490984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:58.491618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:58.491900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:58.492175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:58.498684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:58.500131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:22:58.512894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:22:58.513834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:22:58.514696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 560/1104] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:22:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:22:58.520781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:58.620229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d70cea-b3ce-4a57-bae1-1839890566d3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:58.655580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:58.656199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:58.656474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:58.656740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:58.657679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:58.658547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:58.659993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98ed36-c273-45d6-923f-2fd5823f7496 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:58.667426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:58.672585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 545/1105] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:22:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:22:58.679000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:58.729159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:58.780132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d08b7-ed1e-42c2-a28a-68bae2517d45 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:58.813132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:58.814314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:58.814612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:58.814871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:58.815644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] RBAC: Adding query filter params (role.id=841751d723ec454aae186b4da332a393, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=09fbb54e522c47d89de166682742336f) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:22:58.815949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:58.817405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba63c5-7be2-42a8-8451-f3383e63f5ec None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:58.823064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 561/1106] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:22:58 2022] GET /identity/v3/role_assignments?user.id=09fbb54e522c47d89de166682742336f&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=841751d723ec454aae186b4da332a393 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:59.569934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880993e-25ca-4192-9c0f-3c8f1a782f2b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:59.570454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880993e-25ca-4192-9c0f-3c8f1a782f2b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:59.570454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880993e-25ca-4192-9c0f-3c8f1a782f2b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:59.571834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 546/1107] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:22:59.577153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:59.577682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:59.577903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:59.596174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:22:59.597408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:22:59.665223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 562/1108] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:22:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:22:59.690796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880993e-25ca-4192-9c0f-3c8f1a782f2b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:59.788147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880993e-25ca-4192-9c0f-3c8f1a782f2b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:59.820479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbaf90d-8c18-4130-94a9-932e80e1b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:59.821042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaf90d-8c18-4130-94a9-932e80e1b183 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:59.821356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaf90d-8c18-4130-94a9-932e80e1b183 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:59.821631 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaf90d-8c18-4130-94a9-932e80e1b183 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:59.822762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaf90d-8c18-4130-94a9-932e80e1b183 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:59.824192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaf90d-8c18-4130-94a9-932e80e1b183 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:22:59.825795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 547/1109] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:22:59.831519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:22:59.873785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:22:59.930294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314e9f1-0915-4130-aba0-1d36da83580a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:22:59.963212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd7dfe-4539-4187-b405-b1c5d67f7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:22:59.963822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7dfe-4539-4187-b405-b1c5d67f7458 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:22:59.964103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7dfe-4539-4187-b405-b1c5d67f7458 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:22:59.964376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd7dfe-4539-4187-b405-b1c5d67f7458 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:22:59.965055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd7dfe-4539-4187-b405-b1c5d67f7458 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:22:59.966521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd7dfe-4539-4187-b405-b1c5d67f7458 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:00.068753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 563/1110] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:22:59 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:23:02.090083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8b6ea-cccd-4966-8a94-924ad8567ad6 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:02.090607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8b6ea-cccd-4966-8a94-924ad8567ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:02.090607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8b6ea-cccd-4966-8a94-924ad8567ad6 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:02.091950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 548/1111] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:02.097927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:02.098312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:02.098485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:02.116657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:02.117873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:02.183026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 564/1112] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:02.206259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8b6ea-cccd-4966-8a94-924ad8567ad6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:02.271352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:02.305755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8b6ea-cccd-4966-8a94-924ad8567ad6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:02.338543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:02.339165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:02.339446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:02.339719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] PATH_INFO: `/v3/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:02.341188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] RBAC: Authorizing `identity:get_role(role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:02.342544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:02.343865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 549/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:02 2022] GET /identity/v3/roles/7feebec2e0484087af48f1cb50e6f954 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:02.350157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:02.378030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:02.448371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101277c-b610-4fc0-94ef-5b3a3b388df5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:02.482512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:02.483149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:02.483434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:02.483709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] PATH_INFO: `/v3/users/5bf3df146c3541bfae5670e40e73ea08` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:02.496690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] RBAC: Authorizing `identity:get_user(user_id=5bf3df146c3541bfae5670e40e73ea08)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:02.498175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:02.499780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 565/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:02 2022] GET /identity/v3/users/5bf3df146c3541bfae5670e40e73ea08 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:02.506190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:02.602772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea775a69-60d6-4af4-9254-ca19e909d61f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:02.635250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:02.635845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:02.636121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:02.636388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:02.638900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:02.701755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:02.703189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:02.797276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 550/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:02.803295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:02.886821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:02.901597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed137af-9cdb-420a-b413-7cc8f446c9ac None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:02.934211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ee62-f917-455a-ad75-6b59e4376a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:02.934832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ee62-f917-455a-ad75-6b59e4376a85 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:02.935111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ee62-f917-455a-ad75-6b59e4376a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:02.935384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ee62-f917-455a-ad75-6b59e4376a85 None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:02.936517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ee62-f917-455a-ad75-6b59e4376a85 None admin] RBAC: Authorizing `identity:get_project(project_id=50f0037c482d4504a1db39b4a2995bd9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:02.937984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ee62-f917-455a-ad75-6b59e4376a85 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:02.940195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 566/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:02 2022] GET /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:02.946240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:02.978865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:03.043022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162bd8e-3afe-46ad-89ce-421a3aef1d75 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:03.075816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:03.076417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:03.076694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:03.076964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:03.077777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] RBAC: Adding query filter params (role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=50f0037c482d4504a1db39b4a2995bd9, user.id=5bf3df146c3541bfae5670e40e73ea08) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:03.078079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:03.080081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e06ff5-bca5-4dc6-8e0e-944d33784eb0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:03.085321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 551/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:23:02 2022] GET /identity/v3/role_assignments?user.id=5bf3df146c3541bfae5670e40e73ea08&scope.project.id=50f0037c482d4504a1db39b4a2995bd9&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:05.029721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97088fae-bf18-442a-a00d-53040e61151d None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:05.030246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97088fae-bf18-442a-a00d-53040e61151d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:05.030246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97088fae-bf18-442a-a00d-53040e61151d None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:05.031616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 567/1118] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:05.037266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:05.037670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:05.037851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:05.055706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:05.056897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:05.121923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 552/1119] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:05.145184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97088fae-bf18-442a-a00d-53040e61151d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:05.243417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97088fae-bf18-442a-a00d-53040e61151d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:05.276562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:05.277192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:05.277503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:05.277777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] PATH_INFO: `/v3/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:05.279195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] RBAC: Authorizing `identity:get_role(role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:05.280618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:05.281940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 568/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:05 2022] GET /identity/v3/roles/7feebec2e0484087af48f1cb50e6f954 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:05.288213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:05.315400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:05.385376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b7ef0-44f0-4997-bb5c-22282b7a9046 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:05.418260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:05.418885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:05.419162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:05.419431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] PATH_INFO: `/v3/users/5bf3df146c3541bfae5670e40e73ea08` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:05.420569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] RBAC: Authorizing `identity:get_user(user_id=5bf3df146c3541bfae5670e40e73ea08)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:05.422026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:05.423609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 553/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:05 2022] GET /identity/v3/users/5bf3df146c3541bfae5670e40e73ea08 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:05.429422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:05.452935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:05.525985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4154908-4b72-430a-89ff-db66f0871e98 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:05.559067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa46a2c-354c-4351-8698-0045a59f5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:05.559631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa46a2c-354c-4351-8698-0045a59f5832 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:05.559912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa46a2c-354c-4351-8698-0045a59f5832 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:05.560186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa46a2c-354c-4351-8698-0045a59f5832 None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:05.561341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa46a2c-354c-4351-8698-0045a59f5832 None admin] RBAC: Authorizing `identity:get_project(project_id=50f0037c482d4504a1db39b4a2995bd9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:05.562780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa46a2c-354c-4351-8698-0045a59f5832 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:05.564599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 569/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:05 2022] GET /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:05.570823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:05.662100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:05.667638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f9f14-5c44-45a3-87ac-cb02fb73d34f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:05.699891 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dc730-b220-4775-9496-8a3a115dbc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:05.700538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dc730-b220-4775-9496-8a3a115dbc78 None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:05.700818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dc730-b220-4775-9496-8a3a115dbc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:05.701090 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dc730-b220-4775-9496-8a3a115dbc78 None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9/users/5bf3df146c3541bfae5670e40e73ea08/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:05.703313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dc730-b220-4775-9496-8a3a115dbc78 None admin] RBAC: Authorizing `identity:create_grant(project_id=50f0037c482d4504a1db39b4a2995bd9, user_id=5bf3df146c3541bfae5670e40e73ea08, role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:05.705512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dc730-b220-4775-9496-8a3a115dbc78 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:05.776441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 554/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:23:05 2022] PUT /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9/users/5bf3df146c3541bfae5670e40e73ea08/roles/7feebec2e0484087af48f1cb50e6f954 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:23:07.744296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271726-29fe-472c-8940-e1cf35c8f267 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:07.744825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271726-29fe-472c-8940-e1cf35c8f267 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:07.744825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271726-29fe-472c-8940-e1cf35c8f267 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:07.746211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 570/1124] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:07.751427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:07.752029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:07.752029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:07.770540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:07.771722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:07.836232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 555/1125] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:07.859450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271726-29fe-472c-8940-e1cf35c8f267 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:07.947936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:07.958549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271726-29fe-472c-8940-e1cf35c8f267 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:07.991665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:07.992290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:07.992568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:07.992838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] PATH_INFO: `/v3/roles/7feebec2e0484087af48f1cb50e6f954` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:07.994257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] RBAC: Authorizing `identity:get_role(role_id=7feebec2e0484087af48f1cb50e6f954)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:07.995716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:07.996996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 571/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:07 2022] GET /identity/v3/roles/7feebec2e0484087af48f1cb50e6f954 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:08.003702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:08.084679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:08.099096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957d1a6-df3b-4f70-a0ea-d1be875391a1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:08.132833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:08.133493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:08.133771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:08.134039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] PATH_INFO: `/v3/users/5bf3df146c3541bfae5670e40e73ea08` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:08.135174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] RBAC: Authorizing `identity:get_user(user_id=5bf3df146c3541bfae5670e40e73ea08)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:08.136661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:08.138952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 556/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:08 2022] GET /identity/v3/users/5bf3df146c3541bfae5670e40e73ea08 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:08.144130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:08.241077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be4246-f98e-417a-b2ff-8346cb5b2267 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:08.274880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:08.275422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:08.275701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:08.275971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:08.301815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:08.341630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:08.343067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:08.437472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 572/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:08.444049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:08.541189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7b21e-6af6-43bd-b8ce-1a9f6b559e8c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:08.574655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171e1fd-c59b-4b26-bb55-6642a92de3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:08.575275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e1fd-c59b-4b26-bb55-6642a92de3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:08.575555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e1fd-c59b-4b26-bb55-6642a92de3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:08.575828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e1fd-c59b-4b26-bb55-6642a92de3c6 None admin] PATH_INFO: `/v3/projects/50f0037c482d4504a1db39b4a2995bd9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:08.576965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171e1fd-c59b-4b26-bb55-6642a92de3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=50f0037c482d4504a1db39b4a2995bd9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:08.578438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171e1fd-c59b-4b26-bb55-6642a92de3c6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:08.580274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 557/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:08 2022] GET /identity/v3/projects/50f0037c482d4504a1db39b4a2995bd9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:08.586191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:08.642552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:08.683480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefd848-1a51-4f81-9446-1cdfee0606ab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:08.716416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:08.717012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:08.717316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:08.717590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:08.718363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] RBAC: Adding query filter params (role.id=7feebec2e0484087af48f1cb50e6f954, scope.project.id=50f0037c482d4504a1db39b4a2995bd9, user.id=5bf3df146c3541bfae5670e40e73ea08) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:08.718668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:08.720753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d6ba4c-b354-4ede-b78d-93bf7d93a140 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:08.726333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 573/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:23:08 2022] GET /identity/v3/role_assignments?user.id=5bf3df146c3541bfae5670e40e73ea08&scope.project.id=50f0037c482d4504a1db39b4a2995bd9&role.id=7feebec2e0484087af48f1cb50e6f954 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:10.765725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4b67-26d2-4a0f-957e-43253cad7575 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:10.766362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4b67-26d2-4a0f-957e-43253cad7575 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:10.766362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc4b67-26d2-4a0f-957e-43253cad7575 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:10.767626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 558/1131] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:10.772802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:10.773267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:10.773369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:10.792517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:10.793729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:10.866682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 574/1132] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:10.889944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4b67-26d2-4a0f-957e-43253cad7575 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:10.913699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:10.988981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc4b67-26d2-4a0f-957e-43253cad7575 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:11.021187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9c0e8-0056-4c06-82ca-c7082a2cf71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:11.021781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9c0e8-0056-4c06-82ca-c7082a2cf71e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:11.022057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9c0e8-0056-4c06-82ca-c7082a2cf71e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:11.022326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9c0e8-0056-4c06-82ca-c7082a2cf71e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:11.023449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9c0e8-0056-4c06-82ca-c7082a2cf71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:11.024876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9c0e8-0056-4c06-82ca-c7082a2cf71e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:11.026486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 559/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:23:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:11.032671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:11.084161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:11.131236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab9e0c-4aaf-489c-8244-4728ded9eae5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:11.169690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a6bca-5de4-4f24-8ed7-5a2ac3e1d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:11.170429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a6bca-5de4-4f24-8ed7-5a2ac3e1d491 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:11.170814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a6bca-5de4-4f24-8ed7-5a2ac3e1d491 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:11.171189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a6bca-5de4-4f24-8ed7-5a2ac3e1d491 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:11.172053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a6bca-5de4-4f24-8ed7-5a2ac3e1d491 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:11.174635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a6bca-5de4-4f24-8ed7-5a2ac3e1d491 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:11.287215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 575/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:23:11 2022] POST /identity/v3/projects => generated 311 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:23:13.308162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c506-be3d-467e-a0bf-e609f0d74828 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:13.308680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c506-be3d-467e-a0bf-e609f0d74828 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:13.308680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c506-be3d-467e-a0bf-e609f0d74828 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:13.310075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 560/1135] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:13.315666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:13.316043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:13.316220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:13.334615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:13.335795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:13.402063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 576/1136] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:13.426787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c506-be3d-467e-a0bf-e609f0d74828 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:13.515664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:13.526184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c506-be3d-467e-a0bf-e609f0d74828 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:13.558611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053423a-1182-4a25-b5d1-f5285ce6c090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:13.559230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053423a-1182-4a25-b5d1-f5285ce6c090 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:13.559509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053423a-1182-4a25-b5d1-f5285ce6c090 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:13.559781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053423a-1182-4a25-b5d1-f5285ce6c090 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:13.560918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053423a-1182-4a25-b5d1-f5285ce6c090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:13.562380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053423a-1182-4a25-b5d1-f5285ce6c090 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:13.565514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 561/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:13.571151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:13.604384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:13.669326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5480629-8898-4a17-94dd-1ecd82f2d5ee None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:13.703089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479eb4a-4104-4c20-8d6f-771e034aa648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:13.703630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479eb4a-4104-4c20-8d6f-771e034aa648 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:13.704428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479eb4a-4104-4c20-8d6f-771e034aa648 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:13.705171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479eb4a-4104-4c20-8d6f-771e034aa648 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:13.705893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479eb4a-4104-4c20-8d6f-771e034aa648 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:13.707322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479eb4a-4104-4c20-8d6f-771e034aa648 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:13.765171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 577/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:23:13 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:23:15.769232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7b03-7592-46b8-8f7e-b0db37ff353a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:15.769775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7b03-7592-46b8-8f7e-b0db37ff353a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:15.769887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea7b03-7592-46b8-8f7e-b0db37ff353a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:15.771154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 562/1139] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:15.776157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:15.776525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:15.776705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:15.794884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:15.796047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:15.860874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 578/1140] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:15.883610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea7b03-7592-46b8-8f7e-b0db37ff353a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:15.983482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea7b03-7592-46b8-8f7e-b0db37ff353a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:16.016973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:16.017625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:16.017909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:16.018184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:16.028110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:23:16.033551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:16.034699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:23:16.035737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 563/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:23:16.040783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:16.061099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:16.137134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a723c-8121-4798-9fcb-f19656a21105 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:16.170844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:16.171474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:16.171782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:16.172058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:16.173082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:16.173338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:16.174728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:16.182900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 579/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:16.188738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:16.208349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:16.287990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd057a7-7292-4382-8624-1329307b01a4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:16.320985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:16.321645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:16.321928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:16.322209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] PATH_INFO: `/v3/users/0bc1bea5af1540edb987e83e06b0330a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:16.335513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] RBAC: Authorizing `identity:get_user(user_id=0bc1bea5af1540edb987e83e06b0330a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:16.336959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:16.338601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 564/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:16 2022] GET /identity/v3/users/0bc1bea5af1540edb987e83e06b0330a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:16.344312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:16.426265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:16.444510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd4459-8686-4045-ba2a-49f14e9d68b8 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:16.478403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:16.479019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:16.479301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:16.479574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:16.545018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:16.546442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:16.623714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:16.642262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 580/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:16.648449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:16.723229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:16.745678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee382931-268f-401d-9e67-5ffb3c8288c1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:16.778550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618479d-590b-4e66-a36c-dd84d606b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:16.779181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618479d-590b-4e66-a36c-dd84d606b01d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:16.779462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618479d-590b-4e66-a36c-dd84d606b01d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:16.779733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618479d-590b-4e66-a36c-dd84d606b01d None admin] PATH_INFO: `/v3/projects/13c6e8aac52e436490409b6a3f960f59` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:16.780879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618479d-590b-4e66-a36c-dd84d606b01d None admin] RBAC: Authorizing `identity:get_project(project_id=13c6e8aac52e436490409b6a3f960f59)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:16.782394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1618479d-590b-4e66-a36c-dd84d606b01d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:16.784130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 565/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:16 2022] GET /identity/v3/projects/13c6e8aac52e436490409b6a3f960f59 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:16.790223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:16.887136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee102a8-b51b-490f-93ed-b98a3f714c71 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:16.919725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:16.920327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:16.920615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:16.920890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:16.921695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=13c6e8aac52e436490409b6a3f960f59, user.id=0bc1bea5af1540edb987e83e06b0330a) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:16.922005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:16.924046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119043c-3949-465a-a8c2-88a1bcb8d5e7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:16.929371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 581/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:23:16 2022] GET /identity/v3/role_assignments?user.id=0bc1bea5af1540edb987e83e06b0330a&scope.project.id=13c6e8aac52e436490409b6a3f960f59&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:18.880533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3914-2764-4ab0-9503-ac0d29b95e90 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:18.881062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3914-2764-4ab0-9503-ac0d29b95e90 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:18.881062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f3914-2764-4ab0-9503-ac0d29b95e90 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:18.882497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 566/1147] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:18.887543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:18.887962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:18.888091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:18.905685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:18.906931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:18.948756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:18.972138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 582/1148] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:23:18.994546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f3914-2764-4ab0-9503-ac0d29b95e90 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:19.094068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f3914-2764-4ab0-9503-ac0d29b95e90 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:19.129097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:19.129741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:19.130018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:19.130289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:19.140051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:23:19.145818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:19.146875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:23:19.147876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 567/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:23:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:23:19.152832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:19.253250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7091c80-fa03-4beb-a9e9-7e44b83b2b27 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:19.286560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:19.287163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:19.287442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:19.287713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:19.288414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:19.288713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:19.290115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:19.297961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 583/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:19.303762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:19.308088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:19.402336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205dc4b-97a1-4ca6-9fbc-53196ee3c1ac None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:19.434799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:19.435417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:19.435696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:19.435967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] PATH_INFO: `/v3/users/0bc1bea5af1540edb987e83e06b0330a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:19.437107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] RBAC: Authorizing `identity:get_user(user_id=0bc1bea5af1540edb987e83e06b0330a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:19.439121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:19.440689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 568/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:19 2022] GET /identity/v3/users/0bc1bea5af1540edb987e83e06b0330a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:19.446403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:19.486240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:19.545268 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe6cef-c186-45aa-8b12-e423ae9c4711 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:19.577736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c723c2-5321-40c6-9537-43f1bb1c9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:19.578353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c723c2-5321-40c6-9537-43f1bb1c9576 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:19.578630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c723c2-5321-40c6-9537-43f1bb1c9576 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:19.580427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c723c2-5321-40c6-9537-43f1bb1c9576 None admin] PATH_INFO: `/v3/projects/13c6e8aac52e436490409b6a3f960f59` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:19.581599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c723c2-5321-40c6-9537-43f1bb1c9576 None admin] RBAC: Authorizing `identity:get_project(project_id=13c6e8aac52e436490409b6a3f960f59)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:19.583029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c723c2-5321-40c6-9537-43f1bb1c9576 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:19.584659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 584/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:19 2022] GET /identity/v3/projects/13c6e8aac52e436490409b6a3f960f59 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:19.590477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:19.657995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:19.686835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ebf4d-122e-48aa-bddf-137ce5d65bfb None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:19.720017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcedd4-8f0e-4cfa-a56a-f13aa807090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:19.720668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcedd4-8f0e-4cfa-a56a-f13aa807090a None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:19.720947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcedd4-8f0e-4cfa-a56a-f13aa807090a None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:19.721242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcedd4-8f0e-4cfa-a56a-f13aa807090a None admin] PATH_INFO: `/v3/projects/13c6e8aac52e436490409b6a3f960f59/users/0bc1bea5af1540edb987e83e06b0330a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:19.723161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcedd4-8f0e-4cfa-a56a-f13aa807090a None admin] RBAC: Authorizing `identity:create_grant(project_id=13c6e8aac52e436490409b6a3f960f59, user_id=0bc1bea5af1540edb987e83e06b0330a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:19.724587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcedd4-8f0e-4cfa-a56a-f13aa807090a None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:19.805098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 569/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:23:19 2022] PUT /identity/v3/projects/13c6e8aac52e436490409b6a3f960f59/users/0bc1bea5af1540edb987e83e06b0330a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:23:21.765744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f28a9-d5f4-4470-9015-f8ef4a50024e None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:21.766327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f28a9-d5f4-4470-9015-f8ef4a50024e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:21.767001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f28a9-d5f4-4470-9015-f8ef4a50024e None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:21.768234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 585/1154] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:21.773311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:21.773713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:21.773885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:21.791616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:21.792791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:21.857514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 570/1155] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:21.881129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f28a9-d5f4-4470-9015-f8ef4a50024e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:21.979101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f28a9-d5f4-4470-9015-f8ef4a50024e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:21.986699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:22.013396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:22.014023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:22.014304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:22.014575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:22.024838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:23:22.029925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:22.030985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:23:22.032537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 586/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:23:22.038447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:22.098171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:22.134359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873bb2c-06c9-4421-be95-7c62db075cca None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:22.168851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:22.169496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:22.169772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:22.170037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:22.170748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:22.171043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:22.172402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:22.180397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 571/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:22.186130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:22.284574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fb4e1-5de1-4372-97eb-6809b01a114a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:22.317477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:22.318099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:22.318375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:22.318647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] PATH_INFO: `/v3/users/0bc1bea5af1540edb987e83e06b0330a` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:22.319784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] RBAC: Authorizing `identity:get_user(user_id=0bc1bea5af1540edb987e83e06b0330a)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:22.321236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:22.322806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 587/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:22 2022] GET /identity/v3/users/0bc1bea5af1540edb987e83e06b0330a => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:22.328390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:22.426956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dab2d4-2e77-45f3-82af-1c5dc4c4664d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:22.449642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:22.460865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:22.461523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:22.461801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:22.462067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:22.528377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:22.529876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:22.628089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 572/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:22.633966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:22.667962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:22.732993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46b4b-e694-4e29-bb97-19684e2fb862 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:22.765631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708210f9-2dc9-4375-a2e4-fb418c8d5e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:22.766784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708210f9-2dc9-4375-a2e4-fb418c8d5e5e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:22.767320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708210f9-2dc9-4375-a2e4-fb418c8d5e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:22.767415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708210f9-2dc9-4375-a2e4-fb418c8d5e5e None admin] PATH_INFO: `/v3/projects/13c6e8aac52e436490409b6a3f960f59` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:22.768537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708210f9-2dc9-4375-a2e4-fb418c8d5e5e None admin] RBAC: Authorizing `identity:get_project(project_id=13c6e8aac52e436490409b6a3f960f59)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:22.769999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708210f9-2dc9-4375-a2e4-fb418c8d5e5e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:22.771641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 588/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:22 2022] GET /identity/v3/projects/13c6e8aac52e436490409b6a3f960f59 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:22.777658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:22.802394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:22.873951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10947b-7434-43b2-8b75-01084dd01432 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:22.908475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:22.909012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:22.909318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:22.909587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:22.910364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=13c6e8aac52e436490409b6a3f960f59, user.id=0bc1bea5af1540edb987e83e06b0330a) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:22.910668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:22.912104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34faf013-944d-4648-b68a-d182d6d385ac None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:22.917703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 573/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:23:22 2022] GET /identity/v3/role_assignments?user.id=0bc1bea5af1540edb987e83e06b0330a&scope.project.id=13c6e8aac52e436490409b6a3f960f59&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:25.082706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6a62a-aa8d-4b5f-9079-88cccb5b0c2b None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:25.083243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6a62a-aa8d-4b5f-9079-88cccb5b0c2b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:25.083243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6a62a-aa8d-4b5f-9079-88cccb5b0c2b None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:25.084620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 589/1162] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:25.089884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:25.090332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:25.090430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:25.108148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:25.109344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:25.174362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 574/1163] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:23:25.196519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6a62a-aa8d-4b5f-9079-88cccb5b0c2b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:25.295037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6a62a-aa8d-4b5f-9079-88cccb5b0c2b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:25.296506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:25.327869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:25.328465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:25.328889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:25.328889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:25.393731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:25.395170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:25.489454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 590/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:25.495362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:25.513373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:25.590445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a0886-cd0d-46ca-a63c-5894666252da None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:25.623425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:25.624047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:25.624323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:25.624590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:25.631317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:25.632748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-2762d0dc-f41d-476e-8397-8cd2ed56780f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:25.645051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:25.646264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:23:25.647111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 575/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:23:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:23:25.652077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:25.654275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:25.749988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be426c-71e8-4d3d-94d7-13f3fbed3dbb None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:25.783986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:25.784596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:25.784879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:25.785156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:25.785876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:25.786207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:25.787581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b68e8-618b-400e-a5c1-9447b732f6aa None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:25.794529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 591/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Aug 4 03:23:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:27.766617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980a4c7-8251-4c2f-bc73-6139e0113de3 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:27.767142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980a4c7-8251-4c2f-bc73-6139e0113de3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:27.767142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980a4c7-8251-4c2f-bc73-6139e0113de3 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:27.768547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 576/1167] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:27.774200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298741-5510-481b-aeb2-ea18a22ce6bf None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:27.774599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298741-5510-481b-aeb2-ea18a22ce6bf None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:27.774751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a298741-5510-481b-aeb2-ea18a22ce6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:27.792303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7a298741-5510-481b-aeb2-ea18a22ce6bf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:27.793492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7a298741-5510-481b-aeb2-ea18a22ce6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:27.851128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:27.858217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 592/1168] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:27.881493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980a4c7-8251-4c2f-bc73-6139e0113de3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:27.958479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:27.980289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980a4c7-8251-4c2f-bc73-6139e0113de3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:28.019551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41308ee-8d5c-48ab-9918-798a464d8f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:28.020505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41308ee-8d5c-48ab-9918-798a464d8f2d None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:28.020968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41308ee-8d5c-48ab-9918-798a464d8f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:28.021275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41308ee-8d5c-48ab-9918-798a464d8f2d None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:28.022297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41308ee-8d5c-48ab-9918-798a464d8f2d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:28.023743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41308ee-8d5c-48ab-9918-798a464d8f2d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:28.080873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 577/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Thu Aug 4 03:23:27 2022] POST /identity/v3/domains => generated 267 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:23:30.109049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c133c1-0b9a-4b32-92fb-1bdd45afa8a1 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:30.109648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c133c1-0b9a-4b32-92fb-1bdd45afa8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:30.109648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c133c1-0b9a-4b32-92fb-1bdd45afa8a1 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:30.111508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 593/1170] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:23:30.116640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:30.117010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:30.117180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:30.135004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:30.136177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:30.201464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 578/1171] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:30.225794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c133c1-0b9a-4b32-92fb-1bdd45afa8a1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:30.323315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c133c1-0b9a-4b32-92fb-1bdd45afa8a1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:30.356458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5441967-36b5-401c-b692-d2a3dfbd028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:30.357085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5441967-36b5-401c-b692-d2a3dfbd028f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:30.357393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5441967-36b5-401c-b692-d2a3dfbd028f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:30.357669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5441967-36b5-401c-b692-d2a3dfbd028f None admin] PATH_INFO: `/v3/domains/1e0840bd03804ff498856d458da8a1b4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:30.367834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5441967-36b5-401c-b692-d2a3dfbd028f None admin] RBAC: Authorizing `identity:get_domain(domain_id=1e0840bd03804ff498856d458da8a1b4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:30.369270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5441967-36b5-401c-b692-d2a3dfbd028f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:30.379709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 594/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Thu Aug 4 03:23:30 2022] GET /identity/v3/domains/1e0840bd03804ff498856d458da8a1b4 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:30.379814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:30.442212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:30.475744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f9076-6208-4d5e-a5d6-6beeffab4174 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:30.509516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06590a1b-36e5-41be-b2be-de5462a7778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:30.510124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06590a1b-36e5-41be-b2be-de5462a7778d None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:30.510406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06590a1b-36e5-41be-b2be-de5462a7778d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:30.510680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06590a1b-36e5-41be-b2be-de5462a7778d None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:30.511358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06590a1b-36e5-41be-b2be-de5462a7778d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:30.512787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06590a1b-36e5-41be-b2be-de5462a7778d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:30.588923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 579/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:23:30 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:23:32.589696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca869-e179-4599-b507-609fbbf2a1ea None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:32.590208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca869-e179-4599-b507-609fbbf2a1ea None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:32.590208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeca869-e179-4599-b507-609fbbf2a1ea None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:32.591527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 595/1174] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:32.596857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:32.597248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:32.597435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:32.615865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:32.617027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:32.681572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 580/1175] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:32.705530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca869-e179-4599-b507-609fbbf2a1ea None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:32.761468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:32.805609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeca869-e179-4599-b507-609fbbf2a1ea None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:32.838457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9d4d-57f8-43d1-a7e3-c0f7e0c48e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:32.839080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9d4d-57f8-43d1-a7e3-c0f7e0c48e77 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:32.839361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9d4d-57f8-43d1-a7e3-c0f7e0c48e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:32.839635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9d4d-57f8-43d1-a7e3-c0f7e0c48e77 None admin] PATH_INFO: `/v3/domains/1e0840bd03804ff498856d458da8a1b4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:32.840834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad9d4d-57f8-43d1-a7e3-c0f7e0c48e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1e0840bd03804ff498856d458da8a1b4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:32.842257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad9d4d-57f8-43d1-a7e3-c0f7e0c48e77 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:32.843825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 596/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Thu Aug 4 03:23:32 2022] GET /identity/v3/domains/1e0840bd03804ff498856d458da8a1b4 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:32.849518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:32.897979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:32.947132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e54f7-01ec-484f-aeef-ede4be018c9c None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:32.979882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8aaf9-34b8-44d5-bbb0-c0c5648f2d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:32.980487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8aaf9-34b8-44d5-bbb0-c0c5648f2d0e None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:32.980765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8aaf9-34b8-44d5-bbb0-c0c5648f2d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:32.981035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8aaf9-34b8-44d5-bbb0-c0c5648f2d0e None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:32.981747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8aaf9-34b8-44d5-bbb0-c0c5648f2d0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:32.983174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8aaf9-34b8-44d5-bbb0-c0c5648f2d0e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:33.056143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 581/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:23:32 2022] POST /identity/v3/users => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:23:35.076765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973bc19-3478-4119-b745-4d7240050e52 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:35.077352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973bc19-3478-4119-b745-4d7240050e52 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:35.077491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973bc19-3478-4119-b745-4d7240050e52 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:35.078842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 597/1178] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:35.084057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:35.084460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:35.084649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:35.102998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:35.104173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:35.169829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 582/1179] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:35.193713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973bc19-3478-4119-b745-4d7240050e52 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:35.293474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973bc19-3478-4119-b745-4d7240050e52 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:35.316317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:35.326376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:35.327056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:35.327285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:35.327559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:35.338790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:23:35.343663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:35.344749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:23:35.345633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 598/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:23:35.350961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:35.385224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:35.447494 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa879ce-9644-4d09-a23f-b3cc9aca7916 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:35.479340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:35.479940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:35.480213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:35.480478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:35.481181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:35.481507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:35.482864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:35.491094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 583/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:35.496889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:35.594903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9ac6f-be11-4d93-bd13-a06971404b5e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:35.628266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:35.628893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:35.629177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:35.629483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] PATH_INFO: `/v3/users/10deaa564f5e4e84b54bef10c5c81b73` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:35.642671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] RBAC: Authorizing `identity:get_user(user_id=10deaa564f5e4e84b54bef10c5c81b73)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:35.644115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:35.645749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 599/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:35 2022] GET /identity/v3/users/10deaa564f5e4e84b54bef10c5c81b73 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:35.651462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:35.743345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:35.746741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e74a82-a703-4679-9f8e-c1e503eea650 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:35.780079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:35.780683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:35.780959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:35.781244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:35.845258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:35.846640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:35.935925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:35.941144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 584/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:35.948625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:35.988977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:36.044076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ef970-0f59-4b75-b0e2-628768b7999c None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:36.082169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220decca-1132-467e-9544-e3d920e4dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:36.083238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220decca-1132-467e-9544-e3d920e4dc7d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:36.083538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220decca-1132-467e-9544-e3d920e4dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:36.083814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220decca-1132-467e-9544-e3d920e4dc7d None admin] PATH_INFO: `/v3/projects/a1346102e3e647e2b806f3dec57c5550` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:36.084968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220decca-1132-467e-9544-e3d920e4dc7d None admin] RBAC: Authorizing `identity:get_project(project_id=a1346102e3e647e2b806f3dec57c5550)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:36.086433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220decca-1132-467e-9544-e3d920e4dc7d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:36.088083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 600/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:35 2022] GET /identity/v3/projects/a1346102e3e647e2b806f3dec57c5550 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:36.094374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:36.191251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91e31d-6096-42c3-99c6-17ca0e4acee6 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:36.223617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:36.224223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:36.224498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:36.224762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:36.225574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=a1346102e3e647e2b806f3dec57c5550, user.id=10deaa564f5e4e84b54bef10c5c81b73) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:36.225877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:36.227299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf2bb5-56bf-4198-9d7e-3de7c23004a7 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:36.232579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 585/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:23:36 2022] GET /identity/v3/role_assignments?user.id=10deaa564f5e4e84b54bef10c5c81b73&scope.project.id=a1346102e3e647e2b806f3dec57c5550&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:38.191582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc7be8-3cda-4d8f-a898-01952fd407b7 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:38.192033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc7be8-3cda-4d8f-a898-01952fd407b7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:38.192141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc7be8-3cda-4d8f-a898-01952fd407b7 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:38.193515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 601/1186] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:38.198511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:38.198896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:38.199059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:38.217127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:38.218319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:38.272052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:38.281763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 586/1187] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:38.304082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc7be8-3cda-4d8f-a898-01952fd407b7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:38.402989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc7be8-3cda-4d8f-a898-01952fd407b7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:38.413449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:38.436245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:38.436818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:38.437095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:38.437400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:38.448214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:23:38.453020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:38.454169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:23:38.455293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 602/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:23:38.460184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:38.555246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703492f6-3354-44bf-b6a8-cc4997d12b48 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:38.587868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:38.588465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:38.588740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:38.589006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:38.589736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:38.590030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:38.591390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:38.599174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 587/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:38.604844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:38.702800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabe65-cd44-4374-bd3d-4151e3fd3cfa None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:38.735923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:38.736545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:38.736825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:38.737098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] PATH_INFO: `/v3/users/10deaa564f5e4e84b54bef10c5c81b73` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:38.738262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] RBAC: Authorizing `identity:get_user(user_id=10deaa564f5e4e84b54bef10c5c81b73)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:38.739698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:38.741272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 603/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:38 2022] GET /identity/v3/users/10deaa564f5e4e84b54bef10c5c81b73 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:38.746940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:38.801386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:38.851265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d47bf-4de7-4f29-ba38-668d39e2584d None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:38.901218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928b644-bb88-41bf-abff-dc76a0c0e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:38.901218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928b644-bb88-41bf-abff-dc76a0c0e847 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:38.901218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928b644-bb88-41bf-abff-dc76a0c0e847 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:38.901218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928b644-bb88-41bf-abff-dc76a0c0e847 None admin] PATH_INFO: `/v3/projects/a1346102e3e647e2b806f3dec57c5550` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:38.901595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928b644-bb88-41bf-abff-dc76a0c0e847 None admin] RBAC: Authorizing `identity:get_project(project_id=a1346102e3e647e2b806f3dec57c5550)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:38.901595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928b644-bb88-41bf-abff-dc76a0c0e847 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:38.901595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 588/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:38 2022] GET /identity/v3/projects/a1346102e3e647e2b806f3dec57c5550 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:38.901848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:38.935436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:38.992633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00787ec-8109-46ed-9e35-1c8dc6fd4d0d None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:39.026254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6830bc-aa6a-49b1-88ca-ec0378ee83a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:39.026909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6830bc-aa6a-49b1-88ca-ec0378ee83a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:39.027190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6830bc-aa6a-49b1-88ca-ec0378ee83a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:39.027463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6830bc-aa6a-49b1-88ca-ec0378ee83a7 None admin] PATH_INFO: `/v3/projects/a1346102e3e647e2b806f3dec57c5550/users/10deaa564f5e4e84b54bef10c5c81b73/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:39.029419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6830bc-aa6a-49b1-88ca-ec0378ee83a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1346102e3e647e2b806f3dec57c5550, user_id=10deaa564f5e4e84b54bef10c5c81b73, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:39.030852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6830bc-aa6a-49b1-88ca-ec0378ee83a7 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:39.096135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 604/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:23:38 2022] PUT /identity/v3/projects/a1346102e3e647e2b806f3dec57c5550/users/10deaa564f5e4e84b54bef10c5c81b73/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:23:41.099163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7be1ce-c754-44c3-9076-445b9d798be4 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:41.099784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7be1ce-c754-44c3-9076-445b9d798be4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:41.099784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7be1ce-c754-44c3-9076-445b9d798be4 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:41.101061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 589/1193] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:23:41.106237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:41.106634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:41.106781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:41.124346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:23:41.126312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:23:41.193078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 605/1194] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:23:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:23:41.217037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7be1ce-c754-44c3-9076-445b9d798be4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:41.315124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7be1ce-c754-44c3-9076-445b9d798be4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:41.336808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:41.347949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:41.348569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:41.348846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:41.349118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:41.359496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:23:41.365251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:23:41.371076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:23:41.372021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 590/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:23:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:23:41.372900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:41.400512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:41.472547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97dbc2f-ea72-49ad-b556-b588d4113bfe None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:41.506832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:41.507449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:41.507730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:41.508003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:41.508711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:41.509013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:41.510419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:41.518407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 606/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:23:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:41.524278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:41.621817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7548-0d45-49ef-bb72-4494aebb0dc1 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:41.654668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:41.655288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:41.655566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:41.655836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] PATH_INFO: `/v3/users/10deaa564f5e4e84b54bef10c5c81b73` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:41.657734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] RBAC: Authorizing `identity:get_user(user_id=10deaa564f5e4e84b54bef10c5c81b73)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:41.659178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:41.660757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 591/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Aug 4 03:23:41 2022] GET /identity/v3/users/10deaa564f5e4e84b54bef10c5c81b73 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:41.666411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:41.754559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:41.763730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53073990-70c3-49ce-9204-1179c0d48789 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:41.795717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:41.796323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:41.796601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:41.796872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:41.862929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:41.864370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:41.951060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:41.960999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 607/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:23:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:23:41.966859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:42.016316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:42.065169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef03b46-707f-4eb8-85c6-c75320896eab None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:42.097502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becc9da-97a7-414b-9855-1d224d6f8771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:42.098119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becc9da-97a7-414b-9855-1d224d6f8771 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:42.098396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becc9da-97a7-414b-9855-1d224d6f8771 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:42.098663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becc9da-97a7-414b-9855-1d224d6f8771 None admin] PATH_INFO: `/v3/projects/a1346102e3e647e2b806f3dec57c5550` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:42.099789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becc9da-97a7-414b-9855-1d224d6f8771 None admin] RBAC: Authorizing `identity:get_project(project_id=a1346102e3e647e2b806f3dec57c5550)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:42.101256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becc9da-97a7-414b-9855-1d224d6f8771 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:42.102886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 592/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Aug 4 03:23:41 2022] GET /identity/v3/projects/a1346102e3e647e2b806f3dec57c5550 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:23:42.109276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:23:42.207191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409d226-aa5f-4560-ba1d-b2183714559a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:23:42.239509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:23:42.240106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:23:42.240381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:23:42.240648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:23:42.241445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=a1346102e3e647e2b806f3dec57c5550, user.id=10deaa564f5e4e84b54bef10c5c81b73) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:23:42.241749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:23:42.243172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8cef4-cd4e-49f3-81ea-d00d68ff5a63 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:23:42.248763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 608/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:23:42 2022] GET /identity/v3/role_assignments?user.id=10deaa564f5e4e84b54bef10c5c81b73&scope.project.id=a1346102e3e647e2b806f3dec57c5550&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:23:52.092672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=41 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:23:52.246208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=73 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:52.014898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f99a82-f838-46b0-8538-ffbd0ff5c3ec None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:52.015478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f99a82-f838-46b0-8538-ffbd0ff5c3ec None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:52.015478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f99a82-f838-46b0-8538-ffbd0ff5c3ec None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:52.016811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 593/1201] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:24:52.022702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:52.023155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:52.023254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:52.042551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:24:52.043778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:24:52.109857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 609/1202] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:24:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:24:52.134475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f99a82-f838-46b0-8538-ffbd0ff5c3ec None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:52.238345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f99a82-f838-46b0-8538-ffbd0ff5c3ec None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:52.271790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:52.272351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:52.272625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:52.272893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:52.283265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:52.284796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:24:52.298525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:52.299579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:52.300536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:24:52.300536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:24:52.300536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:52.300536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:24:52.300536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 594/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:52.305963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:52.405393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b90b2a-5707-450c-8cb8-77e4bc6909b2 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:52.432990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:52.438679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:52.439930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:52.440212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:52.440485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:52.441172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:52.441505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:52.442895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d069c01-90e5-4fc7-b83f-6410a01f994a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:52.454025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 610/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:52.460493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:52.521543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:52.568905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d3162-60db-4edd-9298-1b19e7ef5a3c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:52.602189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8444c7-e4e8-4322-8e9e-276c1cee27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:52.602800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8444c7-e4e8-4322-8e9e-276c1cee27ef None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:52.603082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8444c7-e4e8-4322-8e9e-276c1cee27ef None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:52.603437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8444c7-e4e8-4322-8e9e-276c1cee27ef None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:52.604109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8444c7-e4e8-4322-8e9e-276c1cee27ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:52.605582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8444c7-e4e8-4322-8e9e-276c1cee27ef None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:52.736422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 595/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Aug 4 03:24:52 2022] POST /identity/v3/users => generated 251 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:24:54.768114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75e54-1225-4d1f-8774-018265d3e726 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:54.768640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75e54-1225-4d1f-8774-018265d3e726 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:54.768640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75e54-1225-4d1f-8774-018265d3e726 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:54.770062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 611/1206] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:24:54.775391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:54.775887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:54.775988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:54.794974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:24:54.796173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:24:54.861434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 596/1207] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:24:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:24:54.884797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75e54-1225-4d1f-8774-018265d3e726 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:54.988406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75e54-1225-4d1f-8774-018265d3e726 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.024132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.024701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.025037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.025352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:55.036650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:24:55.041536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:55.042678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:24:55.043695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 612/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:24:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:55.049637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:55.088920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:55.146616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb2a3b5-2a8b-48be-9581-d46ff23bb143 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.180399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.181018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.181328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.181604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:55.182317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:55.182616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:55.183983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:55.192694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 597/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:55.198589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:55.248457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:55.297114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a349e4-3ace-4ab6-8000-5af01a7e09e7 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.331096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.331723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.332003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.332272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:55.343723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:55.348056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:24:55.360216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:55.365725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:55.366799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:24:55.366799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:24:55.366799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:55.366799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:24:55.366799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 613/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:55.368120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:55.467664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:55.470604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27251-32ca-45a5-a064-8e344309080c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.503302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.503910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.504192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.504464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:55.505149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:55.505481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:55.506852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:55.518874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 598/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:55.524904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:55.628834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48d97e-7b15-40d0-a4d4-77a23ffa93df None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.630375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:55.662473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.663100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.663376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.663646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:55.670783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:55.672220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:24:55.683285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:55.684517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:24:55.685690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 614/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:55.690573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:55.791311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff9092-d2cd-4d2e-a366-6129e5144d34 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.825333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.825939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.826215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.826484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:55.827272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:55.827573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:55.829003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:55.847269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 599/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Aug 4 03:24:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:55.853295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:55.955794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d757a8-18ac-43ee-9c4a-06f2ba66efef None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:55.988403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:55.989014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:55.989325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:55.989602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:56.020477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:56.057884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:56.059408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:56.158788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 615/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:24:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:24:56.164755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:56.183550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:56.263295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d64246-570c-4d8a-b543-fd38d4de1d49 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:56.297043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:56.297801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:56.298075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:56.298389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:56.309360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:56.311104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:24:56.325870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:56.326980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:56.327926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:24:56.327926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:24:56.327926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:56.327926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:24:56.327926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 600/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:56.333607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:56.394603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:56.432142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a7599-03f2-4803-9c18-94b28911c9c4 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:56.465517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:56.466128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:56.466405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:56.466672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:56.467351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:56.467649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:56.469007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:56.482158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 616/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:56.487850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:56.556462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:56.587062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3afa6-1e1f-4126-b3c5-07b8495bb5c2 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:56.619630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:56.620271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:56.620553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:56.620827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:56.627694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:56.629147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:24:56.641327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:56.642799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:24:56.643669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 601/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:56.648897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:56.750187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22716f2-8107-4285-8ce8-a4003b23a1f9 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:56.773602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:56.784935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:56.785576 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:56.785866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:56.786130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:56.786890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:56.787179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:56.788653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab229d04-05ec-47d3-abbd-a3bc8b8b7252 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:56.800337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 617/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:56.806416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:56.906086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7de4fd-1f3a-473e-8e21-64b7e7bdf7d3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:56.933286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:56.939484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:56.940091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:56.940369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:56.940636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:56.941449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=ce4b77895c73479c9811fa2fd6d7ed71) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:56.941758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:56.943207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91bc12-e0e6-414a-9a81-d6c1607d3f6e None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:56.948413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 602/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:24:56 2022] GET /identity/v3/role_assignments?user.id=ce4b77895c73479c9811fa2fd6d7ed71&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:58.913124 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd3cc-2a53-443a-94d4-df4cfc964281 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:58.913697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd3cc-2a53-443a-94d4-df4cfc964281 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:58.913697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd3cc-2a53-443a-94d4-df4cfc964281 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:58.915057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 618/1220] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:24:58.920045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:58.920425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:58.920596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:58.938351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:24:58.940297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:24:59.006547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 603/1221] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:24:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:24:59.028946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd3cc-2a53-443a-94d4-df4cfc964281 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:59.131347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd3cc-2a53-443a-94d4-df4cfc964281 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:59.164900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:59.165482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:59.165761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:59.166032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:59.176985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:24:59.181928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:59.183042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:24:59.183977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 619/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:59.189719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:59.287358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa434974-614f-47ed-b62d-7548f94c0f70 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:59.310557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:59.321518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:59.322136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:59.322411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:59.322671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:59.323383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] RBAC: Adding query filter params (name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:59.323682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:59.325052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:59.332864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 604/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:59.339055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:59.384142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:59.439804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9ed7e-6873-4769-856a-8e47015e307e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:59.472345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:59.473747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:59.474027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:59.474303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:59.484263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:59.486306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:24:59.500624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:59.507967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:59.509036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:24:59.509036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:24:59.509036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:24:59.509036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:24:59.509036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 620/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:59.509404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:59.605987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d565a2-407f-49c0-a403-862858f1e39c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:59.639636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:59.640284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:59.640564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:59.640832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:59.641563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:59.641851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:59.643948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:59.655393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 605/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:24:59.661901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:59.757663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:59.764038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f476b2-e164-4973-bb57-5d34d669396f None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:59.797564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:59.798202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:59.798530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:59.799488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:59.806878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:59.808420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:24:59.820852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:24:59.821957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:24:59.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 621/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:24:59.827977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:24:59.874887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:24:59.928036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f7474-cfe3-449a-888e-28879ad55839 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:24:59.961794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:24:59.962461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:24:59.962743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:24:59.963015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:24:59.963798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:24:59.964102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:24:59.965578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:24:59.982046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 606/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:24:59.987768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:00.089286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89aa6a4-e4c6-4f37-a1bb-0450f352491a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:00.122296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:00.122924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:00.123203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:00.123475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:00.135029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:00.136501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:00.150537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:00.151653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:00.152569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:00.152569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:00.152569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:00.152569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:00.152569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 622/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:00.157438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:00.255771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607af81-66cc-4522-8fb5-6f7725d9f399 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:00.258126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:00.288514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:00.289125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:00.289437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:00.289711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:00.290398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:00.290700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:00.292066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:00.303671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 607/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:00.309311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:00.320451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:00.414509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe592e51-3132-47b1-81d1-da34360e542b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:00.448018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:00.448629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:00.448906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:00.449174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:00.456160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:00.457647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:25:00.470807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:00.471885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:00.473018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 623/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:00.477932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:00.577504 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca55bef-1ab0-4407-82be-5091f1e9d804 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:00.610019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:00.610625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:00.611462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:00.611732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:00.612468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:00.612771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:00.614450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba56c3-6cf9-4d4b-aebc-e6470fc6fb72 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:00.627159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 608/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:25:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:00.633166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:00.697917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:00.731713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2eecf-56dd-4583-9b4b-e82d1a091bd0 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:00.765332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260ba43-d2b0-4632-b758-68312cb197c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:00.765973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ba43-d2b0-4632-b758-68312cb197c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:00.766254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ba43-d2b0-4632-b758-68312cb197c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:00.766531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ba43-d2b0-4632-b758-68312cb197c4 None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/ce4b77895c73479c9811fa2fd6d7ed71/roles/b94a012ed1954740858f831257bb8811` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:00.781064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260ba43-d2b0-4632-b758-68312cb197c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=ce4b77895c73479c9811fa2fd6d7ed71, role_id=b94a012ed1954740858f831257bb8811)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:00.782540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260ba43-d2b0-4632-b758-68312cb197c4 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:00.842819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 624/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:25:00 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/ce4b77895c73479c9811fa2fd6d7ed71/roles/b94a012ed1954740858f831257bb8811 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:25:02.823502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de613-1e8d-41ae-a35d-d67e5098214d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:02.824053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de613-1e8d-41ae-a35d-d67e5098214d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:02.824053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de613-1e8d-41ae-a35d-d67e5098214d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:02.825423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 609/1233] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:02.830932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:02.831315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:02.831487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:02.850326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:02.851495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:02.917498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 625/1234] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:02.941061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8de613-1e8d-41ae-a35d-d67e5098214d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:02.965814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:03.041140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8de613-1e8d-41ae-a35d-d67e5098214d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:03.073749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73524f-8977-4447-b996-782876cf950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:03.074305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73524f-8977-4447-b996-782876cf950f None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:03.074579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73524f-8977-4447-b996-782876cf950f None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:03.075007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73524f-8977-4447-b996-782876cf950f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:03.085499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73524f-8977-4447-b996-782876cf950f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-5a73524f-8977-4447-b996-782876cf950f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:25:03.090429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:03.091512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:03.092721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 610/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Aug 4 03:25:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:03.098288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:03.144722 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:03.198672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb1b7a-d6ce-4e39-a5ce-55cfa1dfabd8 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:03.231943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:03.232559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:03.232856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:03.233115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:03.233861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] RBAC: Adding query filter params (name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:03.234165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:03.235544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:03.243221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 626/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Aug 4 03:25:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:03.249279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73524f-8977-4447-b996-782876cf950f None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:03.332756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:03.348566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73524f-8977-4447-b996-782876cf950f None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:03.382000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:03.382627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:03.382905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:03.383174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:03.393472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:03.394933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:03.409800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:03.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:03.411917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:03.411917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:03.411917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:03.411917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:03.411917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 611/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:03.416756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:03.516557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b672bc-37f2-45e7-bd2a-482fde7a090e None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:03.526023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:03.550294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:03.551001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:03.551279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:03.551547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:03.552224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:03.552520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:03.553910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:03.566253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 627/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:03.572040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:03.670272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edc31f-a304-43c6-a376-29abf2468f4c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:03.704173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:03.704867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:03.705731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:03.706008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:03.708671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:03.713763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:03.715215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:25:03.726303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:03.727394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:03.728831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 612/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:03.733566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:03.833049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725fa3e6-de9b-48a1-ae6a-9a7584c810ab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:03.866561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:03.867169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:03.867452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:03.867727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:03.868513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:03.868826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:03.870327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:03.887989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 628/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Aug 4 03:25:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:03.894299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:03.991946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a584d-4ccd-4feb-af42-55926aa6e3b9 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:04.024949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:04.025630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:04.025942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:04.026223 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:04.086065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:04.092580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:04.094155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:04.192057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 613/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:25:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 03:25:04.197839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:04.225416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:04.297931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f1736-8396-4821-aab2-e5c812750146 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:04.331494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:04.332125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:04.332407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:04.332682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:04.342414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:04.343859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:04.358007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:04.364342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:04.365315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:04.365315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:04.365315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:04.365315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:04.365315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 629/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:04.365586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:04.457968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:04.463605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff32525-0625-47b8-8cee-51aa8a510082 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:04.495528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:04.496143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:04.496424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:04.496693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:04.497415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:04.497702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:04.499069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:04.510934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 614/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:04.516639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:04.600843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:04.622693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085775d1-5db3-4ab3-962b-81df6a5b99a5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:04.657295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:04.657965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:04.658288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:04.658560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:04.665082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:04.666570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:25:04.678447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:04.679512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:04.680572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 630/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:04.685541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:04.784474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd269-89f7-4099-b9e9-ae1e22a5eca0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:04.818388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:04.819002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:04.819283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:04.822713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:04.822713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:04.822713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:04.822713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be694ab9-a986-42e5-95ce-1d83b22a73fe None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:04.833838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 615/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:04.839916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:04.940150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c846-9a59-4de0-9420-b741d202f4ad None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:04.974217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:04.974831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:04.975110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:04.975460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:04.976256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] RBAC: Adding query filter params (role.id=b94a012ed1954740858f831257bb8811, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=ce4b77895c73479c9811fa2fd6d7ed71) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:04.976566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:04.978040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288758c8-0e13-4bd3-aac1-c74a5e3cfcaa None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:04.983876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 631/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:25:04 2022] GET /identity/v3/role_assignments?user.id=ce4b77895c73479c9811fa2fd6d7ed71&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=b94a012ed1954740858f831257bb8811 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:07.030619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc95e-ca8e-4e79-9f90-47d2928bbce7 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.031165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc95e-ca8e-4e79-9f90-47d2928bbce7 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.031165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc95e-ca8e-4e79-9f90-47d2928bbce7 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.032515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 616/1247] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:07.038257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.038697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.038838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.045282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:07.057120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:07.058606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:07.123492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 632/1248] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:07.147926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dc95e-ca8e-4e79-9f90-47d2928bbce7 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:07.166497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:07.248270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dc95e-ca8e-4e79-9f90-47d2928bbce7 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:07.282555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:07.283193 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.283607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.283882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.293753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:25:07.298569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:07.299741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:07.300826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 617/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:07.305992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:07.404743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546479-e280-42d3-82f1-98f3d298f228 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:07.433443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:07.438783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:07.439403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.439687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.439961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.441238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:07.441636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:07.443185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:07.451017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 633/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:07.457452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:07.532838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:07.554804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66ecb1-e0aa-4719-a637-5f3038ffdba5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:07.587253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:07.587876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.588154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.588426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.599444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:07.600901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:07.615277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:07.616555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:07.618192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:07.618192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:07.618192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:07.618192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:07.618192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 618/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:07.625014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:07.730048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1808b0-b4ac-4df3-bac9-cfaa009648ab None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:07.763961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:07.764577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.764862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.765138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.765855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:07.766162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:07.767544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:07.778951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 634/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:07.784862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:07.884548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded1efd-af36-4b59-8d6d-456500d8f380 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:07.910152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:07.917679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:07.918305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:07.918585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:07.918857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:07.926724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:07.928162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:25:07.939018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:07.940098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:07.941071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 619/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:07.946202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:08.001819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:08.045677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bf847-bb6a-4fcc-94eb-79f40ae51dcc None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:08.079963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:08.080574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:08.080850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:08.081118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:08.082088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:08.082392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:08.083852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:08.101702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 635/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Aug 4 03:25:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:08.107749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:08.210669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74be8a-4a68-4c17-b963-173f18a0f59c None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:08.244286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:08.244899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:08.245182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:08.245496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:08.299777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:08.311883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:08.323601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:08.409942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 620/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:25:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:25:08.416872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:08.515668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3588ca-6a65-4dbb-9795-deabe599eabf None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:08.529185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:08.549908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:08.550456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:08.550735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:08.551008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:08.560815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:08.562337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:08.577325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:08.578360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:08.579452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:08.579452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:08.579452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:08.579452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:08.579452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 636/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:08.585098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:08.676452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:08.688432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a397287-ba02-4ddd-8e6c-09f5bf083886 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:08.722411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:08.723028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:08.723315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:08.723688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:08.724327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:08.724628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:08.726040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:08.737186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 621/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:08.742977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:08.846657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e58b4-1070-4fec-acb4-e2a518ad16ca None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:08.880787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:08.881447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:08.881745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:08.882002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:08.888453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:08.889925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:25:08.902181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:08.903388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:08.904772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 637/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:08.909332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:09.011448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d40de2-493c-46f5-b0c3-e16f65b63445 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:09.043337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:09.043955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:09.044237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:09.044511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:09.045305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:09.045611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:09.047059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adef7ac-70ee-440e-81d1-02527ed54872 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:09.059390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 622/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:25:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:25:09.065439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:09.091211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:09.163655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb0762-6638-4e51-b324-591cbaf541db None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:09.198177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:09.201011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:09.201011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:09.201011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:09.201011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=ce4b77895c73479c9811fa2fd6d7ed71) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:09.201011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:09.201856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e7fe-37a8-4eaf-8a87-7b032c34dd29 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:09.207116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 638/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:25:09 2022] GET /identity/v3/role_assignments?user.id=ce4b77895c73479c9811fa2fd6d7ed71&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:11.190731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071d93a-e187-4660-b56a-067cb9479575 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:11.191189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071d93a-e187-4660-b56a-067cb9479575 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:11.191274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071d93a-e187-4660-b56a-067cb9479575 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:11.192805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 623/1261] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:11.197995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:11.198415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:11.198555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:11.217128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:11.218360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:11.284277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 639/1262] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:11.306463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071d93a-e187-4660-b56a-067cb9479575 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:11.317392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:11.406101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071d93a-e187-4660-b56a-067cb9479575 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:11.439016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:11.439578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:11.439855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:11.440121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:11.450110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:25:11.454976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:11.456042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:11.456931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 624/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:25:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:11.462653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:11.481372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:11.565738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec5dcc-6070-4a3c-b0e3-92c531d6c4c2 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:11.599578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:11.600190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:11.600468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:11.600738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:11.601583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:11.601889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:11.603254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:11.613135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 640/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:25:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:11.619046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:11.686141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:11.719023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeab55c-d55e-4b8c-ac64-d6f3c4c693a5 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:11.753069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550603-8481-409a-b254-86b94b188921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:11.753729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550603-8481-409a-b254-86b94b188921 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:11.754014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550603-8481-409a-b254-86b94b188921 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:11.754284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67550603-8481-409a-b254-86b94b188921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:11.765571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550603-8481-409a-b254-86b94b188921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:11.767063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67550603-8481-409a-b254-86b94b188921 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-67550603-8481-409a-b254-86b94b188921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:11.781972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:11.784760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:11.785933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:11.785933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:11.785933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:11.785933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:11.785933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 625/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:25:11.788953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:11.869814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:11.890488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f7d12-0e85-44d9-961e-6193f47eba00 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:11.924882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:11.925529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:11.925813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:11.926088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:11.927277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:11.927578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:11.928957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:11.940383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 641/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:11.946741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550603-8481-409a-b254-86b94b188921 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:12.045577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67550603-8481-409a-b254-86b94b188921 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:12.064787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:12.079058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:12.079686 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:12.079961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:12.080227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:12.087647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:12.089133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:25:12.100323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:12.103583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:12.104739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 626/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:12.107446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:12.209823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97158686-3a68-41e9-aa6e-d2c3af82a923 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:12.243744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:12.244361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:12.244643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:12.244917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:12.245776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:12.246082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:12.247577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:12.257078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:12.265555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 642/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Aug 4 03:25:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:12.271617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:12.373731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ada93-de03-49fe-a6bf-2c52a80d5c73 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:12.406463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:12.407092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:12.407370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:12.407635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:12.418791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:12.420390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:12.435248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:12.442759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:12.443835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:12.443835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:12.443835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:12.443835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:12.443835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 627/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 03:25:12.444173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:12.544881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebde5-06ec-4d2c-8a32-2a7f95fb1700 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:12.579312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:12.579923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:12.580199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:12.580511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:12.581229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:12.581525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:12.583468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:12.594465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 643/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:12.600169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:12.638467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:12.705528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8827-9eab-4bd1-8c23-b6b8b7797ff0 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:12.737674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:12.738288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:12.738564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:12.739343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:12.745821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:12.747266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:25:12.758926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:12.759903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:12.760803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 628/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:12.766118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:12.825142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:12.865901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67a95c-a9c8-4427-b1e6-7d28d6ec3192 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:12.899887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:12.900438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:12.900722 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:12.901002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:12.901774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:12.902080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:12.903534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde803fb-cb4a-431a-bb87-fda8e83ea66f None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:12.915480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 644/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:12.921465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:13.012489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:13.021140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e370e8a-3b60-4e4b-a0d1-34480c445f06 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:13.055739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c045c7-c57c-4482-9b5e-da632ff60d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:13.056449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c045c7-c57c-4482-9b5e-da632ff60d37 None admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:13.056725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c045c7-c57c-4482-9b5e-da632ff60d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:13.056994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c045c7-c57c-4482-9b5e-da632ff60d37 None admin] PATH_INFO: `/v3/projects/cdc5a6c10712453980a6cf3426011060/users/ce4b77895c73479c9811fa2fd6d7ed71/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:13.058976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c045c7-c57c-4482-9b5e-da632ff60d37 None admin] RBAC: Authorizing `identity:create_grant(project_id=cdc5a6c10712453980a6cf3426011060, user_id=ce4b77895c73479c9811fa2fd6d7ed71, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:13.060411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c045c7-c57c-4482-9b5e-da632ff60d37 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:13.134749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 629/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Aug 4 03:25:12 2022] PUT /identity/v3/projects/cdc5a6c10712453980a6cf3426011060/users/ce4b77895c73479c9811fa2fd6d7ed71/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:25:15.134834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1231c-52ef-4004-83f0-043f6a5b90d9 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:15.135513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1231c-52ef-4004-83f0-043f6a5b90d9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:15.135513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1231c-52ef-4004-83f0-043f6a5b90d9 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:15.136833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 645/1274] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:15.142475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:15.143005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:15.143104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:15.163981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:15.167742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:15.239019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 630/1275] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:15.263278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1231c-52ef-4004-83f0-043f6a5b90d9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:15.375370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1231c-52ef-4004-83f0-043f6a5b90d9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:15.394811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:15.412491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:15.413216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:15.413500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:15.413785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:15.423512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 03:25:15.428710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:15.429881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:15.430952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 646/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Aug 4 03:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:15.435976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:15.515410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:15.536107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152f988-f586-4ed3-886b-dfed96f0ab6a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:15.569922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:15.570562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:15.570841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:15.571110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:15.571822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:15.572124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:15.573652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:15.581454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 631/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Aug 4 03:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:15.588238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:15.688481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c0de0-7c40-4682-9598-d5230cbf640b None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:15.721953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:15.722585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:15.722867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:15.723139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:15.733507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:15.734972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:15.749434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:15.750830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:15.751812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:15.751812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:15.751812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:15.751812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:15.751812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 647/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:15.757415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:15.856566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58d2fc-855e-45b9-9d6b-615d79cb3be4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:15.890123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:15.890729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:15.891010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:15.891280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:15.891965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:15.892265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:15.893672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:15.905069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 632/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:15.911187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:15.944548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:16.012842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75f027-402f-43ef-85fd-282f5081f4b5 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:16.045251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:16.045881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:16.046160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:16.046429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:16.054332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:16.059545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 03:25:16.066848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:16.068946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:16.070126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 648/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:16.074616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:16.078418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:16.171456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b96e1f-8002-4977-94b2-6961376dee4b None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:16.205707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:16.206260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:16.206536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:16.206802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:16.208028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:16.208341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:16.209803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:16.227164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 633/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Aug 4 03:25:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:16.233189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:16.323599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:16.333443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77effa-34c1-4c48-896d-e11693274e92 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:16.365998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:16.372748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:16.372748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:16.372748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:16.434359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:16.435814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:16.522168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:16.531938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 649/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Aug 4 03:25:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 03:25:16.537940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:16.597644 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:16.636015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488fb4a-e040-4aaa-8358-6e765b50e4b4 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:16.674422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:16.675056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:16.675334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:16.675606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:16.686037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:16.687484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 03:25:16.702432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:16.703461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:16.704841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 03:25:16.704841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 03:25:16.704841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 03:25:16.704841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:16.704841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 634/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:16.709832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:16.808934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d270911-f5d8-4683-8582-ac1ada705c98 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:16.842305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:16.842915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:16.843195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:16.843469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:16.844153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:16.844454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:16.845864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:16.857080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 650/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Aug 4 03:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:16.862908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:16.960643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef6f82-88af-4b7a-a6a9-042e0ecc4d14 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:16.973971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:16.992756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:16.993407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:16.993685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:16.993951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:17.000225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:17.001962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 03:25:17.013286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:17.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: ERROR keystone.server.flask.application Aug 04 03:25:17.019849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 635/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Aug 4 03:25:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:17.021064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:17.070897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:17.119138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa5795-ff9b-4869-a4f7-98b7f307e84a None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:17.154430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:17.155044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:17.155328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:17.155603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:17.156351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:17.156653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:17.158120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384d999-bf12-481d-8467-a8505315a8f3 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:17.170370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 651/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Aug 4 03:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:17.177491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:17.279479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99787a5d-3fef-432b-8153-a7de98070185 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:17.312772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:17.313436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:17.313714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:17.313980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:17.314751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] RBAC: Adding query filter params (role.id=2388a99b8be348d1a010f30ed34e35f8, scope.project.id=cdc5a6c10712453980a6cf3426011060, user.id=ce4b77895c73479c9811fa2fd6d7ed71) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:17.315052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:17.316620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451509a9-4597-4cae-bcf1-24ee2ed3c435 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:17.328305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 636/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Aug 4 03:25:17 2022] GET /identity/v3/role_assignments?user.id=ce4b77895c73479c9811fa2fd6d7ed71&scope.project.id=cdc5a6c10712453980a6cf3426011060&role.id=2388a99b8be348d1a010f30ed34e35f8 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:19.361334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3140-ea8b-4e68-9b95-6f0d749fe704 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:19.361895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3140-ea8b-4e68-9b95-6f0d749fe704 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:19.361895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c3140-ea8b-4e68-9b95-6f0d749fe704 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:19.363298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 652/1288] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:19.368646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:19.369089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:19.369183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:19.388239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:19.389452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:19.418700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:19.455409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 637/1289] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 03:25:19.477008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3140-ea8b-4e68-9b95-6f0d749fe704 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:19.578383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c3140-ea8b-4e68-9b95-6f0d749fe704 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:19.589398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:19.612391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:19.613125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:19.613434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:19.614337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:19.615244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:19.616597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:19.625892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:19.627082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:19.628112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 653/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:25:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:19.633127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:19.731293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f677d68-ee22-4199-a79a-b2e19b7aa96d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:19.763993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:19.764689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:19.764984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:19.765286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:19.765952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:19.766247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:19.767687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65dce0c-9781-40c5-a609-3e976dfce59b None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:19.773408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 638/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Aug 4 03:25:19 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:19.779284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:19.880192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf065390-9578-491c-a1c6-888ddedcadcc None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:19.913811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:19.914425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:19.914709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:19.914984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] PATH_INFO: `/v3/services` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:19.915663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:19.915962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:19.917370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540f163-cc3f-4519-8de6-ae909405e3e2 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:19.921834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 654/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Aug 4 03:25:19 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:21.918590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5e198-2ed6-4a48-9e75-ff8b2ff3487f None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:21.919129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5e198-2ed6-4a48-9e75-ff8b2ff3487f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:21.919129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5e198-2ed6-4a48-9e75-ff8b2ff3487f None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:21.920505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 639/1293] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:21.925562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdd92d-3ddd-436d-8ba5-d3eab6a3bf84 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:21.925978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdd92d-3ddd-436d-8ba5-d3eab6a3bf84 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:21.926135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdd92d-3ddd-436d-8ba5-d3eab6a3bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:21.944153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7dbdd92d-3ddd-436d-8ba5-d3eab6a3bf84 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:21.945409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7dbdd92d-3ddd-436d-8ba5-d3eab6a3bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:21.975877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:22.011853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 655/1294] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:22.034890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5e198-2ed6-4a48-9e75-ff8b2ff3487f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:22.065874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:22.135495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5e198-2ed6-4a48-9e75-ff8b2ff3487f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:22.170876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3d220-7e30-4bf3-a841-c587b6a145b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:22.171422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3d220-7e30-4bf3-a841-c587b6a145b5 None admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:22.171701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3d220-7e30-4bf3-a841-c587b6a145b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:22.171973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3d220-7e30-4bf3-a841-c587b6a145b5 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:22.173140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3d220-7e30-4bf3-a841-c587b6a145b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:22.174077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3d220-7e30-4bf3-a841-c587b6a145b5 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:22.272578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 640/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Aug 4 03:25:22 2022] POST /identity/v3/services => generated 246 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:25:24.296135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe8d78-da2c-494b-99d8-48f27cd69f6f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:24.296669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe8d78-da2c-494b-99d8-48f27cd69f6f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:24.296669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe8d78-da2c-494b-99d8-48f27cd69f6f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:24.298157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 656/1296] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:24.303843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:24.304278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:24.304472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:24.323074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:24.324274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:24.405064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 641/1297] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:24.428289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe8d78-da2c-494b-99d8-48f27cd69f6f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:24.533700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe8d78-da2c-494b-99d8-48f27cd69f6f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:24.569789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:24.570424 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:24.570712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:24.570984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:24.571602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:24.572985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:24.582113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:24.588757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:24.589855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 657/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:25:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:24.591528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:24.617099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:24.740719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c99ab-ac07-4c5a-a5d0-003ac65d661e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:24.780586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:24.781240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:24.781520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:24.781793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:24.782470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:24.782765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:24.784206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:24.790655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 642/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Aug 4 03:25:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:24.796539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:24.834589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:24.906275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba44c1-f3cf-4fd6-8146-b6a2248f8690 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:24.951684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:24.952309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:24.952594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:24.952869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:24.953675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] RBAC: Adding query filter params (interface=public, service_id=57af8ceb094a4580a50785c4e897ad7c, region_id=RegionOne) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:24.953985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:24.955372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8534f-7df0-446d-b4a7-788261fe8d78 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:24.961927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 658/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Aug 4 03:25:24 2022] GET /identity/v3/endpoints?service_id=57af8ceb094a4580a50785c4e897ad7c&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:25:24.969434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:25.057403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:25.076929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f80ca-d128-4801-b536-f9c7695a4ca3 None admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:25.114364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930dedc-cb87-4e19-b9ea-5b2fefab088d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:25.114973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930dedc-cb87-4e19-b9ea-5b2fefab088d None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:25.115266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930dedc-cb87-4e19-b9ea-5b2fefab088d None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:25.115517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930dedc-cb87-4e19-b9ea-5b2fefab088d None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:25.116167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930dedc-cb87-4e19-b9ea-5b2fefab088d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:25.118137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930dedc-cb87-4e19-b9ea-5b2fefab088d None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:25.124947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 643/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Aug 4 03:25:24 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:25:27.126394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4e19d-aa05-441f-aa99-3d3fe43c0c4c None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:27.126920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4e19d-aa05-441f-aa99-3d3fe43c0c4c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:27.127031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4e19d-aa05-441f-aa99-3d3fe43c0c4c None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:27.128300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 659/1302] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:25:27.133674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:27.134072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:27.134222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:27.152856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:25:27.154127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:25:27.221276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 644/1303] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:25:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 03:25:27.245158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4e19d-aa05-441f-aa99-3d3fe43c0c4c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:27.340477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:27.347112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4e19d-aa05-441f-aa99-3d3fe43c0c4c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:27.381897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:27.382465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:27.382749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:27.383021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:27.383646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:27.385071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 03:25:27.393831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 03:25:27.396104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application with Lock( Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self._enter() Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application return self.creator() Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application created_value = creator( Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: ERROR keystone.server.flask.application Aug 04 03:25:27.399965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 660/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Aug 4 03:25:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 03:25:27.401881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:27.474335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:27.495759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6775db-5619-44f7-9f12-e56ee99cdcf3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:27.530032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:27.530638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:27.530914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:27.531181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] PATH_INFO: `/v3/services` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:27.531843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:25:27.532138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:27.533539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0488770-0145-49e8-b818-04f89f752ff6 None admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:27.538684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 645/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Aug 4 03:25:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:25:27.544705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:25:27.646453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e6b6-d075-4df5-a1c4-e8c94c645d64 None admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:25:27.680191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9270d-41fa-49be-b469-417a28fb0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:25:27.680810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9270d-41fa-49be-b469-417a28fb0326 None admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:25:27.681094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9270d-41fa-49be-b469-417a28fb0326 None admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:25:27.681413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9270d-41fa-49be-b469-417a28fb0326 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:25:27.682119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9270d-41fa-49be-b469-417a28fb0326 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:25:27.683492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9270d-41fa-49be-b469-417a28fb0326 None admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:25:27.768213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 661/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:25:27 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:25:37.535595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=30 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:25:37.763676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:30:53.989678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1f1df-cf01-4b10-ac1e-7c9f243fea37 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:53.995011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1f1df-cf01-4b10-ac1e-7c9f243fea37 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:53.995011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1f1df-cf01-4b10-ac1e-7c9f243fea37 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:53.996962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff53f0-ba18-446f-8ec2-33a175d65507 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:53.997409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff53f0-ba18-446f-8ec2-33a175d65507 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:53.997564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff53f0-ba18-446f-8ec2-33a175d65507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:54.002534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 646/1307] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:30:53 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:30:54.134958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b5ff53f0-ba18-446f-8ec2-33a175d65507 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:30:54.136189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b5ff53f0-ba18-446f-8ec2-33a175d65507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:54.360109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 662/1308] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:30:54.415901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e26c2-93f0-483b-829a-8fc7b9aa3175 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:54.416187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e26c2-93f0-483b-829a-8fc7b9aa3175 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:54.416677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e26c2-93f0-483b-829a-8fc7b9aa3175 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:54.418529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 647/1309] 10.222.0.66 () {54 vars in 965 bytes} [Thu Aug 4 03:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:30:54.424997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e2b6-7263-49b8-b298-6f6b0dd4413f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:54.425594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e2b6-7263-49b8-b298-6f6b0dd4413f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:54.426095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344e2b6-7263-49b8-b298-6f6b0dd4413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:54.545335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-5344e2b6-7263-49b8-b298-6f6b0dd4413f None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:30:54.546973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5344e2b6-7263-49b8-b298-6f6b0dd4413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:54.572873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:30:54.654614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 663/1310] 10.222.0.66 () {58 vars in 1062 bytes} [Thu Aug 4 03:30:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:30:54.663958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e26c2-93f0-483b-829a-8fc7b9aa3175 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:30:54.666125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-127e26c2-93f0-483b-829a-8fc7b9aa3175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:54.825261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e26c2-93f0-483b-829a-8fc7b9aa3175 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:30:54.913935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:30:54.914570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:54.914852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:54.915125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:54.916795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:54.977536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:30:55.036621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:30:55.038396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3d87-83e3-4d27-b1f1-c04b341a636a service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:30:55.061400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 648/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:30:57.408540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72426-bb3c-463c-83f8-dcd70dcbe2f6 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:57.409110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72426-bb3c-463c-83f8-dcd70dcbe2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:57.409110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff72426-bb3c-463c-83f8-dcd70dcbe2f6 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:57.410775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 664/1312] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:30:57.416609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3a9e-cf5b-4c02-98ff-c72ed879d77a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:57.417125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3a9e-cf5b-4c02-98ff-c72ed879d77a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:57.417260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3a9e-cf5b-4c02-98ff-c72ed879d77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:57.499472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ce0d3a9e-cf5b-4c02-98ff-c72ed879d77a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:30:57.500670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ce0d3a9e-cf5b-4c02-98ff-c72ed879d77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:57.599309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 649/1313] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:30:57.638162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff72426-bb3c-463c-83f8-dcd70dcbe2f6 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:30:57.639678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aff72426-bb3c-463c-83f8-dcd70dcbe2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:57.747988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff72426-bb3c-463c-83f8-dcd70dcbe2f6 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:30:57.819108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:30:57.821657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:30:57.822286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:30:57.822573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:30:57.822852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:30:57.824508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:30:57.932160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:30:57.933680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a737-c36e-4c51-8ea7-60fa24d49eef service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:30:57.952632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 665/1314] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:00.378851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714f39d-764b-4b7c-9070-de37aaa5df67 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:00.379389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714f39d-764b-4b7c-9070-de37aaa5df67 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:00.379389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714f39d-764b-4b7c-9070-de37aaa5df67 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:00.380742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 650/1315] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:00.386016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147f77-7d92-4600-87bd-fb3386f610ae None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:00.386415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147f77-7d92-4600-87bd-fb3386f610ae None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:00.386568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9147f77-7d92-4600-87bd-fb3386f610ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:00.452049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f9147f77-7d92-4600-87bd-fb3386f610ae None None] MFA Rules not processed for user `7c569f0a04f74c129f5270c857601f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:00.453161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f9147f77-7d92-4600-87bd-fb3386f610ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:00.503878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:00.548748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 666/1316] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:00.602495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ccecd-7ab0-4311-8b2b-c9b32cf867df None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:00.602775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ccecd-7ab0-4311-8b2b-c9b32cf867df None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:00.603048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40ccecd-7ab0-4311-8b2b-c9b32cf867df None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:00.604458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 651/1317] 10.222.0.66 () {54 vars in 965 bytes} [Thu Aug 4 03:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:00.610319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db85260-06f2-4f4f-885b-377b6ec7bce8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:00.610613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db85260-06f2-4f4f-885b-377b6ec7bce8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:00.610891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db85260-06f2-4f4f-885b-377b6ec7bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:00.685744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6db85260-06f2-4f4f-885b-377b6ec7bce8 None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:00.686950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6db85260-06f2-4f4f-885b-377b6ec7bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:00.761694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 667/1318] 10.222.0.66 () {58 vars in 1062 bytes} [Thu Aug 4 03:31:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:00.770423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ccecd-7ab0-4311-8b2b-c9b32cf867df None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:00.772168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a40ccecd-7ab0-4311-8b2b-c9b32cf867df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:00.781725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:00.874271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40ccecd-7ab0-4311-8b2b-c9b32cf867df None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:00.945565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:00.946164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:00.946438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:00.946703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:00.948301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:01.008989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:01.055165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:01.056542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7751830-5541-4b93-a997-1bc0466b9486 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:01.076792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 652/1319] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:04.382621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8e9c-5b5a-4227-8856-50ef4a019ed6 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:04.383173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8e9c-5b5a-4227-8856-50ef4a019ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:04.383292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c8e9c-5b5a-4227-8856-50ef4a019ed6 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:04.384562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 668/1320] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:04.389807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0e6c-084b-4c7c-b9f1-071e396cb5bf None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:04.390190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0e6c-084b-4c7c-b9f1-071e396cb5bf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:04.390352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c0e6c-084b-4c7c-b9f1-071e396cb5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:04.456481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fc9c0e6c-084b-4c7c-b9f1-071e396cb5bf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:04.457647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc9c0e6c-084b-4c7c-b9f1-071e396cb5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:04.553817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 653/1321] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:04.592620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8e9c-5b5a-4227-8856-50ef4a019ed6 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:04.594173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0a1c8e9c-5b5a-4227-8856-50ef4a019ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:04.645019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:04.697918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c8e9c-5b5a-4227-8856-50ef4a019ed6 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:04.769807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:04.770357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:04.770638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:04.770915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:04.772643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:04.867400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:04.880265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:04.881869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b11f4-7a79-4cc6-934d-e0c17f7fbe87 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:04.899658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 669/1322] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:07.881710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a229c81-edf1-45d9-97cb-29efc5ed96a4 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:07.882262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a229c81-edf1-45d9-97cb-29efc5ed96a4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:07.882262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a229c81-edf1-45d9-97cb-29efc5ed96a4 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:07.883621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 654/1323] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:07.888932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95b30-db1d-43eb-bbc2-0e6c167e17d1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:07.889364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95b30-db1d-43eb-bbc2-0e6c167e17d1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:07.889652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95b30-db1d-43eb-bbc2-0e6c167e17d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:07.955810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-57a95b30-db1d-43eb-bbc2-0e6c167e17d1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:07.956954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-57a95b30-db1d-43eb-bbc2-0e6c167e17d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:08.055991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 670/1324] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:08.093380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a229c81-edf1-45d9-97cb-29efc5ed96a4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:08.094927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0a229c81-edf1-45d9-97cb-29efc5ed96a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:08.116612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:08.216939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a229c81-edf1-45d9-97cb-29efc5ed96a4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:08.297754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:08.298375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:08.298653 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:08.298922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:08.300543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:08.374371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:08.412071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:08.413821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a0c7d4-ef16-4e07-b6f4-f3de20787159 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:08.432129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 655/1325] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:10.511553 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ffc6d-a75e-4bb9-8de8-a7ebaf21c1ab None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:10.512081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ffc6d-a75e-4bb9-8de8-a7ebaf21c1ab None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:10.512081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ffc6d-a75e-4bb9-8de8-a7ebaf21c1ab None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:10.513389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 671/1326] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:10.518452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589198d-6eb7-479b-a4ce-bdeeb6ec8110 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:10.518829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589198d-6eb7-479b-a4ce-bdeeb6ec8110 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:10.519000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589198d-6eb7-479b-a4ce-bdeeb6ec8110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:10.584497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b589198d-6eb7-479b-a4ce-bdeeb6ec8110 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:10.585667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b589198d-6eb7-479b-a4ce-bdeeb6ec8110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:10.681899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 656/1327] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:31:10.718413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ffc6d-a75e-4bb9-8de8-a7ebaf21c1ab None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:10.719953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6f0ffc6d-a75e-4bb9-8de8-a7ebaf21c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:10.781105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:10.825130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ffc6d-a75e-4bb9-8de8-a7ebaf21c1ab None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:10.897740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:10.898355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:10.898634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:10.898907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:10.900518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:11.013418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:11.015824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:11.017304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8898f-402d-4c18-b384-a1bb2ab58999 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:11.036022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 672/1328] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:13.106138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5baadb-2974-43e1-913e-48748770a476 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:13.106700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5baadb-2974-43e1-913e-48748770a476 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:13.106700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5baadb-2974-43e1-913e-48748770a476 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:13.108071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 657/1329] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:13.113950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabde55d-87fe-40c6-9e68-1358c1d197bc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:13.114480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabde55d-87fe-40c6-9e68-1358c1d197bc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:13.114597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabde55d-87fe-40c6-9e68-1358c1d197bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:13.187573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-aabde55d-87fe-40c6-9e68-1358c1d197bc None None] MFA Rules not processed for user `7c569f0a04f74c129f5270c857601f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:13.188778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aabde55d-87fe-40c6-9e68-1358c1d197bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:13.284594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 673/1330] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:13.324137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5baadb-2974-43e1-913e-48748770a476 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:13.325683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ea5baadb-2974-43e1-913e-48748770a476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:13.357704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:13.437563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5baadb-2974-43e1-913e-48748770a476 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:13.511027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:13.511638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:13.511918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:13.512191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:13.513833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:13.596084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:13.632396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:13.633957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54485a2-cbfb-4726-94ab-f47424a7d094 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:13.652343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 658/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:16.070287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92974-4fda-4d4a-83c0-75fe22174758 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:16.070807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92974-4fda-4d4a-83c0-75fe22174758 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:16.070807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92974-4fda-4d4a-83c0-75fe22174758 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:16.072206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 674/1332] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:16.077294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a585-944d-458b-baf8-afbd83f6ed82 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:16.077707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a585-944d-458b-baf8-afbd83f6ed82 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:16.077857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402a585-944d-458b-baf8-afbd83f6ed82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:16.143718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f402a585-944d-458b-baf8-afbd83f6ed82 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:16.144937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f402a585-944d-458b-baf8-afbd83f6ed82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:16.242728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 659/1333] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:16.279915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f92974-4fda-4d4a-83c0-75fe22174758 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:16.281518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d0f92974-4fda-4d4a-83c0-75fe22174758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:16.348964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:16.383282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f92974-4fda-4d4a-83c0-75fe22174758 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:16.456494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:16.457044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:16.457352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:16.457630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:16.459761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:16.568207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:16.569973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb5762-f3f9-4b6a-8ccd-c74dfa7555f3 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:16.579800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:16.588479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 675/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:18.630311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3302e-85c4-4769-9789-6c537ec88219 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:18.631495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3302e-85c4-4769-9789-6c537ec88219 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:18.631495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3302e-85c4-4769-9789-6c537ec88219 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:18.632908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 660/1335] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:18.638209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a852dac-fd97-41c3-8b52-c84b907a3f2b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:18.638595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a852dac-fd97-41c3-8b52-c84b907a3f2b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:18.638759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a852dac-fd97-41c3-8b52-c84b907a3f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:18.705159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6a852dac-fd97-41c3-8b52-c84b907a3f2b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:18.706389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6a852dac-fd97-41c3-8b52-c84b907a3f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:18.801319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 676/1336] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:31:18.842522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3302e-85c4-4769-9789-6c537ec88219 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:18.844049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-38d3302e-85c4-4769-9789-6c537ec88219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:18.884621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:18.946922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3302e-85c4-4769-9789-6c537ec88219 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:19.018455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:19.019066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:19.019347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:19.019621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:19.021301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:19.103702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:19.130866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:19.132313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514709ee-49ce-4ed1-b339-4946d2a8c977 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:19.150215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 661/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:21.749032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d4e02-172a-454a-8f0e-25f31a886f61 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:21.749531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d4e02-172a-454a-8f0e-25f31a886f61 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:21.749640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d4e02-172a-454a-8f0e-25f31a886f61 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:21.751013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 677/1338] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:21.756369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdd940-6e88-4b5f-bd02-c888b9617c96 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:21.756787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdd940-6e88-4b5f-bd02-c888b9617c96 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:21.756917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdd940-6e88-4b5f-bd02-c888b9617c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:21.822032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4ffdd940-6e88-4b5f-bd02-c888b9617c96 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:21.823213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4ffdd940-6e88-4b5f-bd02-c888b9617c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:21.918888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 662/1339] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:21.955148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d4e02-172a-454a-8f0e-25f31a886f61 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:21.956626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f12d4e02-172a-454a-8f0e-25f31a886f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:22.032412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:22.059584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d4e02-172a-454a-8f0e-25f31a886f61 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:22.139516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:22.140323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:22.140605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:22.140877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:22.142642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:22.256327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:22.257881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13877691-58a1-4616-8868-5e14981acc59 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:22.277127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 678/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:26.444708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e8e8d-160a-454a-9135-51c22b16dd81 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:26.445229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e8e8d-160a-454a-9135-51c22b16dd81 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:26.445387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e8e8d-160a-454a-9135-51c22b16dd81 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:26.446875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 663/1341] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:31:26.452330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5a911-3d5a-4a5b-98e0-5c3be14206b5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:26.452761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5a911-3d5a-4a5b-98e0-5c3be14206b5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:26.452888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5a911-3d5a-4a5b-98e0-5c3be14206b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:26.458343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:26.523164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a5b5a911-3d5a-4a5b-98e0-5c3be14206b5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:26.524345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a5b5a911-3d5a-4a5b-98e0-5c3be14206b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:26.622263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 679/1342] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:26.661823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e8e8d-160a-454a-9135-51c22b16dd81 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:26.663346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7a8e8e8d-160a-454a-9135-51c22b16dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:26.712254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:26.768807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e8e8d-160a-454a-9135-51c22b16dd81 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:26.842873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:26.843491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:26.843774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:26.844045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:26.845845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:26.938357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:26.954993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:26.956428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbec6a3-0d07-4015-b275-f27745d9fd1d service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:26.974846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 664/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:30.085742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8922488-b1cd-4719-be72-9f4516b82638 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:30.086262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8922488-b1cd-4719-be72-9f4516b82638 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:30.086262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8922488-b1cd-4719-be72-9f4516b82638 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:30.087642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 680/1344] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:30.092899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16c5b8-da54-4910-90d7-dccb2def4d14 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:30.093371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16c5b8-da54-4910-90d7-dccb2def4d14 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:30.093488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16c5b8-da54-4910-90d7-dccb2def4d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:30.160868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-1a16c5b8-da54-4910-90d7-dccb2def4d14 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:30.162736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1a16c5b8-da54-4910-90d7-dccb2def4d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:30.265768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 665/1345] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:30.303287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8922488-b1cd-4719-be72-9f4516b82638 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:30.304893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e8922488-b1cd-4719-be72-9f4516b82638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:30.396146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:30.413813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8922488-b1cd-4719-be72-9f4516b82638 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:30.489314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:30.489955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:30.490241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:30.490518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:30.492300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:30.609688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:30.611173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c160de8-ffbc-449e-b36c-80b0c4b22cf4 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:30.631932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 681/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:34.957710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864989-4f38-47d0-8efa-e2c5916d2bf9 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:34.958267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864989-4f38-47d0-8efa-e2c5916d2bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:34.958267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25864989-4f38-47d0-8efa-e2c5916d2bf9 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:34.960240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 666/1347] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:34.965999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151d46-ebed-41fe-80e8-b4ca746353a8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:34.966443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151d46-ebed-41fe-80e8-b4ca746353a8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:34.966730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be151d46-ebed-41fe-80e8-b4ca746353a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:34.983507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:35.041123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-be151d46-ebed-41fe-80e8-b4ca746353a8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:35.042447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-be151d46-ebed-41fe-80e8-b4ca746353a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:35.144878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 682/1348] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:35.184620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25864989-4f38-47d0-8efa-e2c5916d2bf9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:35.186221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-25864989-4f38-47d0-8efa-e2c5916d2bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:35.248789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:35.296834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25864989-4f38-47d0-8efa-e2c5916d2bf9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:35.371625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:35.372238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:35.372515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:35.372781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:35.374471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:35.484454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:35.488061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:35.490210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ee5d0-3e67-4706-a75e-6adc1614a48c service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:35.509219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 667/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:40.416252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67271-4ed5-4c08-8377-e9e311f4367e None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:40.416735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67271-4ed5-4c08-8377-e9e311f4367e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:40.417239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f67271-4ed5-4c08-8377-e9e311f4367e None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:40.418787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 683/1350] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:40.424173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261cfb-2301-4dad-9f2b-b83508860cf8 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:40.424599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261cfb-2301-4dad-9f2b-b83508860cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:40.424720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27261cfb-2301-4dad-9f2b-b83508860cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:40.547258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-27261cfb-2301-4dad-9f2b-b83508860cf8 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:40.548440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-27261cfb-2301-4dad-9f2b-b83508860cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:40.674156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 668/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:31:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:45.117990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa02af-293c-4f8a-86a1-0f85c062dcfb None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:45.118520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa02af-293c-4f8a-86a1-0f85c062dcfb None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:45.118520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa02af-293c-4f8a-86a1-0f85c062dcfb None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:45.120347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 684/1352] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:45.125332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57970b-e5cf-4d08-85bc-439479aae489 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:45.126155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57970b-e5cf-4d08-85bc-439479aae489 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:45.126661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57970b-e5cf-4d08-85bc-439479aae489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:45.143233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=59 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:45.207956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:45.232503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-1c57970b-e5cf-4d08-85bc-439479aae489 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:45.233750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1c57970b-e5cf-4d08-85bc-439479aae489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:45.350754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 669/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:31:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:46.304799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c787e1f-63bc-42fa-9b64-93f4fffaa23f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:46.304799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c787e1f-63bc-42fa-9b64-93f4fffaa23f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:46.304799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c787e1f-63bc-42fa-9b64-93f4fffaa23f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:46.304799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 685/1354] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:46.309770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d5ee2-d766-45f9-973b-4dd0b4643693 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:46.310513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d5ee2-d766-45f9-973b-4dd0b4643693 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:46.310947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d5ee2-d766-45f9-973b-4dd0b4643693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:46.390182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-797d5ee2-d766-45f9-973b-4dd0b4643693 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:46.391403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-797d5ee2-d766-45f9-973b-4dd0b4643693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:46.433315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:46.499785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 670/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:31:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:50.026157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68559a63-c69d-43c1-a5fb-50f003d9e6ec None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:50.026666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68559a63-c69d-43c1-a5fb-50f003d9e6ec None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:50.027135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68559a63-c69d-43c1-a5fb-50f003d9e6ec None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:50.029540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 686/1356] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:50.035126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5164b6c-6ad6-4273-b8b9-95b213561ffe None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:50.036632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5164b6c-6ad6-4273-b8b9-95b213561ffe None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:50.037253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5164b6c-6ad6-4273-b8b9-95b213561ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:50.113498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e5164b6c-6ad6-4273-b8b9-95b213561ffe None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:50.114831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e5164b6c-6ad6-4273-b8b9-95b213561ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:50.233674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 671/1357] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:50.291939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9614bb-169c-47e0-8be5-d82bf52f7c25 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:50.292774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9614bb-169c-47e0-8be5-d82bf52f7c25 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:50.293295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9614bb-169c-47e0-8be5-d82bf52f7c25 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:50.294885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 687/1358] 10.222.0.66 () {54 vars in 962 bytes} [Thu Aug 4 03:31:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:50.304461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2e14-7764-4d10-8d27-682b8f2a2269 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:50.304807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2e14-7764-4d10-8d27-682b8f2a2269 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:50.305088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2e14-7764-4d10-8d27-682b8f2a2269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:50.322869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:50.391220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4ace2e14-7764-4d10-8d27-682b8f2a2269 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:50.392509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4ace2e14-7764-4d10-8d27-682b8f2a2269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:50.517825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 672/1359] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Aug 4 03:31:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:50.525902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9614bb-169c-47e0-8be5-d82bf52f7c25 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:50.527421 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cb9614bb-169c-47e0-8be5-d82bf52f7c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:50.702253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9614bb-169c-47e0-8be5-d82bf52f7c25 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:50.782581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:50.825221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:50.825770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:50.826050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:50.826321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:50.827984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:50.935930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:50.937717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db8994-6edc-4857-826f-baf7534761cc service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:50.960273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 688/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:53.190568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ab4d5-9b3c-4fd6-8f9d-e19d25df0a4d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:53.191100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ab4d5-9b3c-4fd6-8f9d-e19d25df0a4d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:53.191100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ab4d5-9b3c-4fd6-8f9d-e19d25df0a4d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:53.192802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 673/1361] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:53.198269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc418735-0a40-4bab-af23-7a4bee3994b6 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:53.198855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc418735-0a40-4bab-af23-7a4bee3994b6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:53.198855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc418735-0a40-4bab-af23-7a4bee3994b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:53.268258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-dc418735-0a40-4bab-af23-7a4bee3994b6 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:53.269416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dc418735-0a40-4bab-af23-7a4bee3994b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:53.312063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:53.374480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 689/1362] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:31:53.418756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ab4d5-9b3c-4fd6-8f9d-e19d25df0a4d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:53.421104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f55ab4d5-9b3c-4fd6-8f9d-e19d25df0a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:53.499078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:53.585371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ab4d5-9b3c-4fd6-8f9d-e19d25df0a4d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:53.710779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:53.711423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:53.711725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:53.712043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:53.713859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:53.745465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:53.812687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:53.814438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ba2cf-15ec-41fa-a513-b1aa9e6e227a service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:53.834866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 674/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:56.334744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5abea-cac7-4ed2-9123-2a8b66aade5f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:56.335251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5abea-cac7-4ed2-9123-2a8b66aade5f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:56.335251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5abea-cac7-4ed2-9123-2a8b66aade5f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:56.336766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 690/1364] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:56.342387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f41fc-5369-45b9-aa15-e56b741462ab None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:56.342892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f41fc-5369-45b9-aa15-e56b741462ab None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:56.342988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f41fc-5369-45b9-aa15-e56b741462ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:56.424041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b70f41fc-5369-45b9-aa15-e56b741462ab None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:56.425384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b70f41fc-5369-45b9-aa15-e56b741462ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:56.532969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:56.541546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 675/1365] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:56.586170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b103bb-8762-49bb-bb48-8672ffd73a89 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:56.586727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b103bb-8762-49bb-bb48-8672ffd73a89 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:56.587348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b103bb-8762-49bb-bb48-8672ffd73a89 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:56.589164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 691/1366] 10.222.0.66 () {54 vars in 962 bytes} [Thu Aug 4 03:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:56.594199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d01a-b604-4ec9-90ee-f23483f656d2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:56.594483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d01a-b604-4ec9-90ee-f23483f656d2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:56.594752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4d01a-b604-4ec9-90ee-f23483f656d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:56.671807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d8e4d01a-b604-4ec9-90ee-f23483f656d2 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:56.672998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d8e4d01a-b604-4ec9-90ee-f23483f656d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:56.804287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 676/1367] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Aug 4 03:31:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:56.812289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b103bb-8762-49bb-bb48-8672ffd73a89 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:56.814121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-68b103bb-8762-49bb-bb48-8672ffd73a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:56.977551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b103bb-8762-49bb-bb48-8672ffd73a89 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:56.999973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:57.098070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:57.099008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:57.099389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:57.099837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:57.101509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:57.206819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:57.209336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255655ea-9aef-4f18-9dbb-50fbb31237a1 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:57.229055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 692/1368] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:58.210383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517295-aec6-49e9-9064-cae8c4aced1d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:58.211170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517295-aec6-49e9-9064-cae8c4aced1d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:58.211170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5517295-aec6-49e9-9064-cae8c4aced1d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:58.212495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 677/1369] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:58.218226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe82e1-0a6a-48a3-82e1-94b44a07f9c4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:58.218709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe82e1-0a6a-48a3-82e1-94b44a07f9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:58.219087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe82e1-0a6a-48a3-82e1-94b44a07f9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:58.253317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:58.290607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a1fe82e1-0a6a-48a3-82e1-94b44a07f9c4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:58.292716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a1fe82e1-0a6a-48a3-82e1-94b44a07f9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:58.396065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 693/1370] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:58.740097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65458d-cfe7-4447-903a-02624bbd2f21 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:58.740097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65458d-cfe7-4447-903a-02624bbd2f21 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:58.740097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65458d-cfe7-4447-903a-02624bbd2f21 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:58.742027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 678/1371] 10.222.0.66 () {54 vars in 964 bytes} [Thu Aug 4 03:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:58.746780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1ecaa-2063-4b4c-b273-3b90b757b818 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:58.747075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1ecaa-2063-4b4c-b273-3b90b757b818 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:58.747354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1ecaa-2063-4b4c-b273-3b90b757b818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:58.828252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6ef1ecaa-2063-4b4c-b273-3b90b757b818 None None] MFA Rules not processed for user `428e53d5cb9e4a2996295b94e364ece4`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:58.829499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6ef1ecaa-2063-4b4c-b273-3b90b757b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:58.881866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:58.910354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 694/1372] 10.222.0.66 () {58 vars in 1061 bytes} [Thu Aug 4 03:31:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:58.918513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65458d-cfe7-4447-903a-02624bbd2f21 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:58.920526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2a65458d-cfe7-4447-903a-02624bbd2f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:58.976505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:59.034118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65458d-cfe7-4447-903a-02624bbd2f21 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:59.108026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:31:59.108651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:59.108935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:59.109225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:59.110849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:59.218219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:31:59.232334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:31:59.233754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d55a2e-56f3-4468-a960-ac8d29b16e29 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:31:59.256915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 679/1373] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:31:59.485552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95da0a5-742c-4d2c-a6ac-3e3e7756d7c4 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:59.485552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95da0a5-742c-4d2c-a6ac-3e3e7756d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:59.485552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95da0a5-742c-4d2c-a6ac-3e3e7756d7c4 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:59.486541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 695/1374] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:31:59.491514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90aff6f-8ea0-40b6-8290-d35a532eb6a1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:31:59.491805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90aff6f-8ea0-40b6-8290-d35a532eb6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:31:59.492190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90aff6f-8ea0-40b6-8290-d35a532eb6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:31:59.571982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b90aff6f-8ea0-40b6-8290-d35a532eb6a1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:31:59.573706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b90aff6f-8ea0-40b6-8290-d35a532eb6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:59.713480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 680/1375] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:31:59.756661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95da0a5-742c-4d2c-a6ac-3e3e7756d7c4 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:31:59.758245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a95da0a5-742c-4d2c-a6ac-3e3e7756d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:31:59.917995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95da0a5-742c-4d2c-a6ac-3e3e7756d7c4 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:31:59.969166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:00.038447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:00.039069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:00.039348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:00.039637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:00.041249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:00.132271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:00.133945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac53c6e-41d3-4a52-93f1-6311c3c20379 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:00.152256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 696/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:02.287484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fe996-65d9-4dda-ba77-5423caa24195 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:02.288008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fe996-65d9-4dda-ba77-5423caa24195 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:02.288008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fe996-65d9-4dda-ba77-5423caa24195 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:02.289427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 681/1377] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:02.294760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9f23f-2998-4c67-b289-c3f6896a8e2e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:02.295165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9f23f-2998-4c67-b289-c3f6896a8e2e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:02.295443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9f23f-2998-4c67-b289-c3f6896a8e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:02.366898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2ef9f23f-2998-4c67-b289-c3f6896a8e2e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:02.368109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2ef9f23f-2998-4c67-b289-c3f6896a8e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:02.390149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:02.493674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 697/1378] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:02.538325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fe996-65d9-4dda-ba77-5423caa24195 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:02.539839 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-de9fe996-65d9-4dda-ba77-5423caa24195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:02.599986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:02.691123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fe996-65d9-4dda-ba77-5423caa24195 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:02.829419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:02.830173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:02.830455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:02.830936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:02.834172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:02.855052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:02.944277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:02.946080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb3b5b-c2e2-4533-8c38-78f55d9d4271 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:02.966928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 682/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:05.156396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246935-778b-4f77-99cd-342be270f0e1 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:05.156950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246935-778b-4f77-99cd-342be270f0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:05.156950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e246935-778b-4f77-99cd-342be270f0e1 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:05.161847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 698/1380] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:05.164607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8d03-a7ef-40f7-a02b-e7446040a3c0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:05.164607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8d03-a7ef-40f7-a02b-e7446040a3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:05.164607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db8d03-a7ef-40f7-a02b-e7446040a3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:05.241499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-66db8d03-a7ef-40f7-a02b-e7446040a3c0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:05.242790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-66db8d03-a7ef-40f7-a02b-e7446040a3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:05.340802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:05.347336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 683/1381] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:05.392339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e246935-778b-4f77-99cd-342be270f0e1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:05.393898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4e246935-778b-4f77-99cd-342be270f0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:05.539164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:05.549813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e246935-778b-4f77-99cd-342be270f0e1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:05.674274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:05.674831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:05.675116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:05.675392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:05.677801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:05.785458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:05.787234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb405a3f-256d-4706-ae31-2cce5435487a service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:05.798486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:05.809010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 699/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:07.955676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2ab7d-a22a-4f2c-9570-7280c18444c5 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:07.956226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2ab7d-a22a-4f2c-9570-7280c18444c5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:07.956226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2ab7d-a22a-4f2c-9570-7280c18444c5 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:07.958271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 684/1383] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:07.963721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa9ce-ba54-48ea-8d77-c473966e2d2b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:07.964169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa9ce-ba54-48ea-8d77-c473966e2d2b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:07.964278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afa9ce-ba54-48ea-8d77-c473966e2d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:08.039873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-98afa9ce-ba54-48ea-8d77-c473966e2d2b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:08.041398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-98afa9ce-ba54-48ea-8d77-c473966e2d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:08.162957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 700/1384] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:08.205741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2ab7d-a22a-4f2c-9570-7280c18444c5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:08.207252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e5f2ab7d-a22a-4f2c-9570-7280c18444c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:08.354956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2ab7d-a22a-4f2c-9570-7280c18444c5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:08.422840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:08.469243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:08.469877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:08.470159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:08.470432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:08.472131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:08.564441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:08.566117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c0df0-ffd7-4ea9-ab60-962279347757 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:08.586737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 685/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:10.966013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b520f-d8bc-4768-a1e2-edce71bcbc05 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:10.966565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b520f-d8bc-4768-a1e2-edce71bcbc05 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:10.966565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b520f-d8bc-4768-a1e2-edce71bcbc05 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:10.968008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 701/1386] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:10.974178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ea87-73d2-4761-af5b-06104dbb3281 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:10.974632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ea87-73d2-4761-af5b-06104dbb3281 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:10.974763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307ea87-73d2-4761-af5b-06104dbb3281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:11.044369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9307ea87-73d2-4761-af5b-06104dbb3281 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:11.045897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9307ea87-73d2-4761-af5b-06104dbb3281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:11.083937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:11.159287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 686/1387] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:11.206344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b520f-d8bc-4768-a1e2-edce71bcbc05 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:11.208188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-337b520f-d8bc-4768-a1e2-edce71bcbc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:11.294132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:11.369810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b520f-d8bc-4768-a1e2-edce71bcbc05 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:11.504389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:11.504937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:11.505321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:11.505602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:11.507253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:11.545870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:11.598877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:11.600511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31e02d-102e-4df2-9884-7f5d91d146bb service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:11.620391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 702/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:13.908719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134024ea-946f-4962-ad34-729c23a4b06e None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:13.909331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134024ea-946f-4962-ad34-729c23a4b06e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:13.909331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134024ea-946f-4962-ad34-729c23a4b06e None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:13.911220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 687/1389] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:13.916432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b3220-4178-4582-8fbf-4d49262b1876 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:13.916818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b3220-4178-4582-8fbf-4d49262b1876 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:13.916993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b3220-4178-4582-8fbf-4d49262b1876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:13.983373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ad7b3220-4178-4582-8fbf-4d49262b1876 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:13.984481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ad7b3220-4178-4582-8fbf-4d49262b1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:14.083484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 703/1390] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:14.128146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134024ea-946f-4962-ad34-729c23a4b06e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:14.130479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-134024ea-946f-4962-ad34-729c23a4b06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:14.279793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134024ea-946f-4962-ad34-729c23a4b06e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:14.296060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:14.394608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:14.395571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:14.395851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:14.396127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:14.397875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:14.487228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:14.489705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb225652-27ff-4661-b8c8-3a522c1e493e service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:14.508272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 688/1391] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:14.619166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75135a3b-dd27-4220-889a-204cf43df898 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:14.619360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75135a3b-dd27-4220-889a-204cf43df898 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:14.619589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75135a3b-dd27-4220-889a-204cf43df898 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:14.620972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 704/1392] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:14.626304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6cc41-3dc9-4414-b29d-ad3392acd415 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:14.626593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6cc41-3dc9-4414-b29d-ad3392acd415 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:14.626926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6cc41-3dc9-4414-b29d-ad3392acd415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:14.652604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:14.703989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0de6cc41-3dc9-4414-b29d-ad3392acd415 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:14.705693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0de6cc41-3dc9-4414-b29d-ad3392acd415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:14.821648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 689/1393] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:14.877643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa93e2-e2a6-44bc-8429-4b30f4445899 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:14.877926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa93e2-e2a6-44bc-8429-4b30f4445899 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:14.878201 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa93e2-e2a6-44bc-8429-4b30f4445899 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:14.880674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 705/1394] 10.222.0.66 () {54 vars in 966 bytes} [Thu Aug 4 03:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:14.885739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0571-22e6-46d2-b6c9-aa7a8a85249d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:14.886025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0571-22e6-46d2-b6c9-aa7a8a85249d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:14.886409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a0571-22e6-46d2-b6c9-aa7a8a85249d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:14.962703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c79a0571-22e6-46d2-b6c9-aa7a8a85249d None None] MFA Rules not processed for user `2e4a8d38d08c4619812d98a544510b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:14.963955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c79a0571-22e6-46d2-b6c9-aa7a8a85249d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:15.005371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:15.042617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 690/1395] 10.222.0.66 () {58 vars in 1063 bytes} [Thu Aug 4 03:32:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:15.050092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa93e2-e2a6-44bc-8429-4b30f4445899 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:15.051661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-52aa93e2-e2a6-44bc-8429-4b30f4445899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:15.064947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:15.159425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa93e2-e2a6-44bc-8429-4b30f4445899 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:15.237054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:15.237714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:15.237999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:15.238277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:15.240143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:15.308404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:15.350927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:15.352376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab66a12-0d8e-4faa-9bba-2c3ef0535289 service glance] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:15.371308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 706/1396] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:16.290268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401fa8a-df75-4a97-8d43-7060fefcaa15 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:16.290746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401fa8a-df75-4a97-8d43-7060fefcaa15 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:16.290849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401fa8a-df75-4a97-8d43-7060fefcaa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:16.359311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c401fa8a-df75-4a97-8d43-7060fefcaa15 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:16.360533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c401fa8a-df75-4a97-8d43-7060fefcaa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:16.450312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 691/1397] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:32:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:16.497768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e46f8c-a48a-4832-9e5b-a1d8320f63c6 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:16.498222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e46f8c-a48a-4832-9e5b-a1d8320f63c6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:16.498503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e46f8c-a48a-4832-9e5b-a1d8320f63c6 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:16.500041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 707/1398] 10.222.0.66 () {54 vars in 943 bytes} [Thu Aug 4 03:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:16.507033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:16.507701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:16.508014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:16.579810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:16.587554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] MFA Rules not processed for user `d647c7fcd5fb490db5302a9ead7ab330`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:16.588749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:16.671093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da476-4ac6-451c-bd5d-b6799c3a6d17 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:16.671381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da476-4ac6-451c-bd5d-b6799c3a6d17 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:16.671657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07da476-4ac6-451c-bd5d-b6799c3a6d17 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:16.673010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 708/1399] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:16.679151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:16.679435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:16.679712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:16.700570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 692/1400] 10.222.0.66 () {58 vars in 1040 bytes} [Thu Aug 4 03:32:16 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:16.710557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:16.720126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f69257-a783-4b21-be5e-4783aef686be None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:16.748788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:16.750040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:16.828349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:16.828966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:16.829265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:16.829543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:16.831177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:16.834202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:16.847086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 709/1401] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:16.892501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:16.893820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:16.919186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:16.920801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98876c1c-cbbd-49bf-affc-640cc0da2c64 service swift] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:16.921709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:16.929716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 693/1402] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 03:32:17.011441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8865f-6039-4815-8ede-eecab62188dc None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:17.125381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:17.125998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:17.126281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:17.126558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:17.128195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:17.147645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:17.227860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:17.229532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127cdfd-e36c-469f-90c2-326f1a556b52 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:17.248646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 710/1403] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:32:19.460240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f33905-0c8a-4015-8229-73f4cad9e2c9 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:19.460240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f33905-0c8a-4015-8229-73f4cad9e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:19.460240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f33905-0c8a-4015-8229-73f4cad9e2c9 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:19.461809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 694/1404] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:19.467408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca04c6-0775-4a4f-9574-f8617c8699c1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:19.467866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca04c6-0775-4a4f-9574-f8617c8699c1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:19.467967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca04c6-0775-4a4f-9574-f8617c8699c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:19.555754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a7ca04c6-0775-4a4f-9574-f8617c8699c1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:19.556975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a7ca04c6-0775-4a4f-9574-f8617c8699c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:19.666140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:19.671473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 711/1405] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:19.718337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f33905-0c8a-4015-8229-73f4cad9e2c9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:19.721616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-77f33905-0c8a-4015-8229-73f4cad9e2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:19.914941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:19.917169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f33905-0c8a-4015-8229-73f4cad9e2c9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:20.066381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:20.067462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:20.069051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:20.069385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:20.071050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:20.195624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:20.197937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0034e-8167-4b75-9f7e-fe61c68ecf5e service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:20.225381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 695/1406] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:22.482562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865ef1-2802-4e4d-8c19-fe71d34950a9 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:22.483073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865ef1-2802-4e4d-8c19-fe71d34950a9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:22.483073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865ef1-2802-4e4d-8c19-fe71d34950a9 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:22.485401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 712/1407] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:32:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:22.490323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925a9e-9825-4063-8cd0-eb194f43d955 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:22.490795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925a9e-9825-4063-8cd0-eb194f43d955 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:22.490886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925a9e-9825-4063-8cd0-eb194f43d955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:22.497834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:22.579515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ee925a9e-9825-4063-8cd0-eb194f43d955 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:22.580875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ee925a9e-9825-4063-8cd0-eb194f43d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:22.721224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 696/1408] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:22.758068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865ef1-2802-4e4d-8c19-fe71d34950a9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:22.759588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-06865ef1-2802-4e4d-8c19-fe71d34950a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:22.956801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865ef1-2802-4e4d-8c19-fe71d34950a9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:23.061312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:23.142138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:23.142766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:23.143044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:23.143316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:23.145148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:23.278559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:23.279866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449517b-392a-4bcf-826d-df7dde3c9209 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:23.301650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 713/1409] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:32.706850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=64 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:33.298020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=64 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:40.815992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44efbe0-54e3-493b-84ea-02e92c6f1262 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:40.816535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44efbe0-54e3-493b-84ea-02e92c6f1262 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:40.816535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44efbe0-54e3-493b-84ea-02e92c6f1262 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:40.818457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 697/1410] 10.222.0.66 () {54 vars in 918 bytes} [Thu Aug 4 03:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:40.823958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea6f3d-1baa-478f-8306-7c3ec59aedef None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:40.824858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea6f3d-1baa-478f-8306-7c3ec59aedef None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:40.824858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea6f3d-1baa-478f-8306-7c3ec59aedef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:40.902180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-51ea6f3d-1baa-478f-8306-7c3ec59aedef None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:40.903381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-51ea6f3d-1baa-478f-8306-7c3ec59aedef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:41.011388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 714/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Aug 4 03:32:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:41.026333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6edf-75e8-44d2-9886-ae58bab916f1 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:41.026622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6edf-75e8-44d2-9886-ae58bab916f1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:41.026930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea6edf-75e8-44d2-9886-ae58bab916f1 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:41.028294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 698/1412] 10.222.0.66 () {54 vars in 918 bytes} [Thu Aug 4 03:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:41.033091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1fa9d-4f04-4aca-9526-6d496741970f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:41.033404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1fa9d-4f04-4aca-9526-6d496741970f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:41.033679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1fa9d-4f04-4aca-9526-6d496741970f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:41.106367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:41.109475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a8a1fa9d-4f04-4aca-9526-6d496741970f None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:41.110666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a8a1fa9d-4f04-4aca-9526-6d496741970f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:41.226506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 715/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Aug 4 03:32:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:43.711642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ef196-d41c-4ec3-9d21-26bc0aaed162 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:43.712233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ef196-d41c-4ec3-9d21-26bc0aaed162 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:43.712233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ef196-d41c-4ec3-9d21-26bc0aaed162 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:43.713615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:43.718796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736495f-bd68-4f81-9d68-ab4a734e7620 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:43.719263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736495f-bd68-4f81-9d68-ab4a734e7620 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:43.719375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736495f-bd68-4f81-9d68-ab4a734e7620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:43.786113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c736495f-bd68-4f81-9d68-ab4a734e7620 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:43.787923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c736495f-bd68-4f81-9d68-ab4a734e7620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:43.841977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:43.884079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:43.926352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ef196-d41c-4ec3-9d21-26bc0aaed162 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:43.928476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-429ef196-d41c-4ec3-9d21-26bc0aaed162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:44.073122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ef196-d41c-4ec3-9d21-26bc0aaed162 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:44.146571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:44.189159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:44.189800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:44.190079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:44.190350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:44.191969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:44.280823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:44.282457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fc81b-b34e-4231-9574-a09318f864df service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:44.301226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 700/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:45.197391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab8c73-911e-48be-8c2e-76e4580e3163 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:45.197862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab8c73-911e-48be-8c2e-76e4580e3163 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:45.197973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab8c73-911e-48be-8c2e-76e4580e3163 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:45.199333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:45.204216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf7d3-8822-4cf6-81f1-906929dcbfe6 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:45.204590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf7d3-8822-4cf6-81f1-906929dcbfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:45.204764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf7d3-8822-4cf6-81f1-906929dcbfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:45.279560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d21cf7d3-8822-4cf6-81f1-906929dcbfe6 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:45.280764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d21cf7d3-8822-4cf6-81f1-906929dcbfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:45.307923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:45.377651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:45.528728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab8c73-911e-48be-8c2e-76e4580e3163 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:45.530220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-96ab8c73-911e-48be-8c2e-76e4580e3163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:45.675961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab8c73-911e-48be-8c2e-76e4580e3163 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:45.716253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:45.788544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:45.789104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:45.789415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:45.789694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:45.791310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:45.879139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:45.881563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b850a0-fff5-4cb5-a740-b05348a7c515 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:45.899185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 718/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:46.782850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7008b-5fc4-45ab-8d52-cc4b26abf8bf None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:46.783419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7008b-5fc4-45ab-8d52-cc4b26abf8bf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:46.783419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7008b-5fc4-45ab-8d52-cc4b26abf8bf None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:46.784827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:46.790273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5746f-f0da-4cec-b8ba-4a3b94fbd2b8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:46.790719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5746f-f0da-4cec-b8ba-4a3b94fbd2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:46.790822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a5746f-f0da-4cec-b8ba-4a3b94fbd2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:46.845516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:46.866635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-03a5746f-f0da-4cec-b8ba-4a3b94fbd2b8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:46.867889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-03a5746f-f0da-4cec-b8ba-4a3b94fbd2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:46.966637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:47.120112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7008b-5fc4-45ab-8d52-cc4b26abf8bf None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:47.121738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b8e7008b-5fc4-45ab-8d52-cc4b26abf8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:47.267102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7008b-5fc4-45ab-8d52-cc4b26abf8bf None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:47.278912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:47.379507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:47.380038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:47.380973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:47.381479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:47.383102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:47.470585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:47.472184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211dd373-5e54-432a-ae53-b455a01e3c81 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:47.491597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 703/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:48.384015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0aa72-ef16-4ef3-9841-d5380134c888 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:48.384582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0aa72-ef16-4ef3-9841-d5380134c888 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:48.384798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0aa72-ef16-4ef3-9841-d5380134c888 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:48.386067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:48.391159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4e8a3-ce33-4de8-997a-6e012105c23d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:48.391557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4e8a3-ce33-4de8-997a-6e012105c23d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:48.391714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4e8a3-ce33-4de8-997a-6e012105c23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:48.411742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:48.467058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4aa4e8a3-ce33-4de8-997a-6e012105c23d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:48.468203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4aa4e8a3-ce33-4de8-997a-6e012105c23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:48.562488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:32:48.715934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0aa72-ef16-4ef3-9841-d5380134c888 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:48.717531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-37d0aa72-ef16-4ef3-9841-d5380134c888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:48.827589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:48.863818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0aa72-ef16-4ef3-9841-d5380134c888 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:48.977103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:48.977674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:48.977956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:48.978231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:48.979857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:49.054802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:49.068501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:49.070130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253a6d2-cf3b-4e38-b391-a3440cf9d3b7 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:49.089426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 721/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:49.949192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c14ade-150c-4709-8745-86d445d1b9c9 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:49.949785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c14ade-150c-4709-8745-86d445d1b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:49.949785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c14ade-150c-4709-8745-86d445d1b9c9 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:49.951127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:49.956139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79d5dd-2225-4163-8519-50a53e4fafd9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:49.956556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79d5dd-2225-4163-8519-50a53e4fafd9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:49.956718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79d5dd-2225-4163-8519-50a53e4fafd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:50.033520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1f79d5dd-2225-4163-8519-50a53e4fafd9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:50.034636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1f79d5dd-2225-4163-8519-50a53e4fafd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:50.131443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:50.289557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c14ade-150c-4709-8745-86d445d1b9c9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:50.291014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-40c14ade-150c-4709-8745-86d445d1b9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:50.374605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:50.436125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c14ade-150c-4709-8745-86d445d1b9c9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:50.555614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:50.555614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:50.555614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:50.556118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:50.557394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:50.606424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:50.648127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:50.649856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a93ea1-0e11-40dc-8a37-ebd74733a8c2 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:50.668595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 706/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:51.554873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff83452-5eeb-4148-8f28-6d60365b715b None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:51.555409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff83452-5eeb-4148-8f28-6d60365b715b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:51.555409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff83452-5eeb-4148-8f28-6d60365b715b None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:51.556715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:51.561790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f91370-d792-4e8e-a044-f8d0d1cfcd5a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:51.562194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f91370-d792-4e8e-a044-f8d0d1cfcd5a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:51.562339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f91370-d792-4e8e-a044-f8d0d1cfcd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:51.638276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f8f91370-d792-4e8e-a044-f8d0d1cfcd5a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:51.639486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f8f91370-d792-4e8e-a044-f8d0d1cfcd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:51.735139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:51.887339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff83452-5eeb-4148-8f28-6d60365b715b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:51.888864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aff83452-5eeb-4148-8f28-6d60365b715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:51.937837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:52.035787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff83452-5eeb-4148-8f28-6d60365b715b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:52.149466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:52.150009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:52.150292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:52.150567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:52.152241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:52.159241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:52.242003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:52.243885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c5c1c-1db1-41fd-9993-ef6a45539096 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:52.261839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 724/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:53.122940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465375-071f-4a85-b2a8-15c0f04e75a3 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:53.123441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465375-071f-4a85-b2a8-15c0f04e75a3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:53.123548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8465375-071f-4a85-b2a8-15c0f04e75a3 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:53.124889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:53.129843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499093af-d698-46da-bfb9-c5e3c99f9e31 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:53.131123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499093af-d698-46da-bfb9-c5e3c99f9e31 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:53.131123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499093af-d698-46da-bfb9-c5e3c99f9e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:53.209037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-499093af-d698-46da-bfb9-c5e3c99f9e31 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:53.210285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-499093af-d698-46da-bfb9-c5e3c99f9e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:53.278341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:53.308466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:53.460942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8465375-071f-4a85-b2a8-15c0f04e75a3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:53.462485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a8465375-071f-4a85-b2a8-15c0f04e75a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:53.480221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:53.606732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8465375-071f-4a85-b2a8-15c0f04e75a3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:53.692526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:53.718183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:53.718788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:53.719064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:53.719333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:53.720922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:53.809421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:53.811044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9acbac-6969-4bcd-b10c-72ea4991e584 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:53.829356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 709/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:54.701461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b98fcf-437d-4f7f-a2de-88a0c04eebd3 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:54.701989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b98fcf-437d-4f7f-a2de-88a0c04eebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:54.702101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b98fcf-437d-4f7f-a2de-88a0c04eebd3 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:54.703418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:54.708822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c6c8-0d20-498c-8ef2-b769bcadf5b2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:54.709326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c6c8-0d20-498c-8ef2-b769bcadf5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:54.709427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7c6c8-0d20-498c-8ef2-b769bcadf5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:54.784385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-06d7c6c8-0d20-498c-8ef2-b769bcadf5b2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:54.785570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-06d7c6c8-0d20-498c-8ef2-b769bcadf5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:54.826745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:54.881902 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:55.035936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b98fcf-437d-4f7f-a2de-88a0c04eebd3 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:55.037615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-59b98fcf-437d-4f7f-a2de-88a0c04eebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:55.187875 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b98fcf-437d-4f7f-a2de-88a0c04eebd3 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:55.242013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:55.301016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:55.301599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:55.302402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:55.302692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:55.304354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:55.393377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:55.394994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193515c-fc96-4844-b67e-b0648e50e85b service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:55.412927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 727/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:56.277009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564e50-b87b-4864-8fd5-2a5a1638d39d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:56.277533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564e50-b87b-4864-8fd5-2a5a1638d39d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:56.277659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a564e50-b87b-4864-8fd5-2a5a1638d39d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:56.279014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:56.283880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348e207-65e9-4e67-98ad-9b7a2b1f1600 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:56.284304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348e207-65e9-4e67-98ad-9b7a2b1f1600 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:56.284438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0348e207-65e9-4e67-98ad-9b7a2b1f1600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:56.370130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0348e207-65e9-4e67-98ad-9b7a2b1f1600 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:56.371339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0348e207-65e9-4e67-98ad-9b7a2b1f1600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:56.373354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:56.468707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:56.622229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564e50-b87b-4864-8fd5-2a5a1638d39d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:56.623662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0a564e50-b87b-4864-8fd5-2a5a1638d39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:56.768467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a564e50-b87b-4864-8fd5-2a5a1638d39d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:56.794809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:56.882840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:56.883369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:56.883646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:56.883913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:56.885588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:56.972970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:56.974665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42121dd-0755-49c9-9baa-5783c2b092dc service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:56.993612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 712/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:32:58.905169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dcded-ce57-4ed7-a1b4-e141ea5bdf15 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:58.905749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dcded-ce57-4ed7-a1b4-e141ea5bdf15 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:58.905749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dcded-ce57-4ed7-a1b4-e141ea5bdf15 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:58.907163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:32:58.912493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872035c-18f1-4791-8ddf-9533fbe92ace None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:58.912894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872035c-18f1-4791-8ddf-9533fbe92ace None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:58.913052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872035c-18f1-4791-8ddf-9533fbe92ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:58.953571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:58.979408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-6872035c-18f1-4791-8ddf-9533fbe92ace None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:32:58.980621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6872035c-18f1-4791-8ddf-9533fbe92ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:59.077475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:32:59.117631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dcded-ce57-4ed7-a1b4-e141ea5bdf15 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:32:59.119428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f63dcded-ce57-4ed7-a1b4-e141ea5bdf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:59.225668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dcded-ce57-4ed7-a1b4-e141ea5bdf15 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:32:59.257594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:32:59.300263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:32:59.300890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:32:59.301178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:32:59.301492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:32:59.303154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:32:59.412591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:32:59.414154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58c5076-a974-45dc-93f3-375d3b6db627 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:32:59.432829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 730/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:01.584501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f47eb-3e2e-4ecd-8917-a3cc9d7c42d4 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:01.585020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f47eb-3e2e-4ecd-8917-a3cc9d7c42d4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:01.585020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f47eb-3e2e-4ecd-8917-a3cc9d7c42d4 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:01.586347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:01.591537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e84675-b71c-419c-9acc-a1146e8eb3a5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:01.591958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e84675-b71c-419c-9acc-a1146e8eb3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:01.592121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e84675-b71c-419c-9acc-a1146e8eb3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:01.659726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-48e84675-b71c-419c-9acc-a1146e8eb3a5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:01.660907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-48e84675-b71c-419c-9acc-a1146e8eb3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:01.674479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:01.759740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:01.798402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f47eb-3e2e-4ecd-8917-a3cc9d7c42d4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:01.799971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-727f47eb-3e2e-4ecd-8917-a3cc9d7c42d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:01.904067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f47eb-3e2e-4ecd-8917-a3cc9d7c42d4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:01.916699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:01.978485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:01.979094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:01.980017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:01.980017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:01.981270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:02.092139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:02.093616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e114bf2-17cc-4c26-be98-12184f1e85e7 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:02.111475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 715/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:04.580332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2436e-ba13-4e8f-994c-4180fc46699f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:04.580832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2436e-ba13-4e8f-994c-4180fc46699f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:04.580832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2436e-ba13-4e8f-994c-4180fc46699f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:04.582209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:04.588058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da359b-518b-4155-b5a0-4f62db16623f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:04.588465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da359b-518b-4155-b5a0-4f62db16623f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:04.588606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da359b-518b-4155-b5a0-4f62db16623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:04.636515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:04.654832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-03da359b-518b-4155-b5a0-4f62db16623f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:04.656065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-03da359b-518b-4155-b5a0-4f62db16623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:04.750619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:04.790273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2436e-ba13-4e8f-994c-4180fc46699f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:04.791844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e2e2436e-ba13-4e8f-994c-4180fc46699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:04.894824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2436e-ba13-4e8f-994c-4180fc46699f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:04.935003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:04.970253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:04.970869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:04.971150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:04.971423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:04.973052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:05.081122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:05.082611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846aad8-20d9-4779-99e7-06bb0dd57c39 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:05.100976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 733/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:07.210154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d93a7-d40c-4007-b584-f080e755c0ff None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:07.210661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d93a7-d40c-4007-b584-f080e755c0ff None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:07.210778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d93a7-d40c-4007-b584-f080e755c0ff None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:07.212042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:07.217182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aeee5-acc3-48d4-a7de-221f7819150a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:07.217620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aeee5-acc3-48d4-a7de-221f7819150a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:07.217764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6aeee5-acc3-48d4-a7de-221f7819150a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:07.283438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-de6aeee5-acc3-48d4-a7de-221f7819150a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:07.284631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-de6aeee5-acc3-48d4-a7de-221f7819150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:07.306374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:07.385617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:33:07.424159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d93a7-d40c-4007-b584-f080e755c0ff None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:07.425659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-740d93a7-d40c-4007-b584-f080e755c0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:07.528564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d93a7-d40c-4007-b584-f080e755c0ff None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:07.548062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:07.602712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:07.603341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:07.603620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:07.603886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:07.605542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:07.713748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:07.715111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cdff7-1b87-4bea-a9e0-fe2dfe9287c7 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:07.733061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 718/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:09.803323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e97c6-627f-4a3e-8555-75b07c03f828 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:09.803796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e97c6-627f-4a3e-8555-75b07c03f828 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:09.803890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e97c6-627f-4a3e-8555-75b07c03f828 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:09.805263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:09.810550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b6c36-1d5a-4d7f-b99c-b3f450f5909f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:09.810983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b6c36-1d5a-4d7f-b99c-b3f450f5909f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:09.811113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b6c36-1d5a-4d7f-b99c-b3f450f5909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:09.876657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:09.879380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-576b6c36-1d5a-4d7f-b99c-b3f450f5909f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:09.880912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-576b6c36-1d5a-4d7f-b99c-b3f450f5909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:10.096982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:33:10.135845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e97c6-627f-4a3e-8555-75b07c03f828 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:10.137436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a11e97c6-627f-4a3e-8555-75b07c03f828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:10.241328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e97c6-627f-4a3e-8555-75b07c03f828 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:10.289887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:10.314634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:10.315248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:10.315530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:10.315803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:10.318120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:10.436880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:10.438366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ce50b-722f-40c7-a23c-be9bf516c521 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:10.456928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 736/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:12.513371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5d52-2f1c-4524-946a-0c0c90034141 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:12.513930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5d52-2f1c-4524-946a-0c0c90034141 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:12.513930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5d52-2f1c-4524-946a-0c0c90034141 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:12.515337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:12.520431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fe2b-c9f0-4c15-bb86-311e1305edf2 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:12.520848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fe2b-c9f0-4c15-bb86-311e1305edf2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:12.520994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7fe2b-c9f0-4c15-bb86-311e1305edf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:12.586797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6ae7fe2b-c9f0-4c15-bb86-311e1305edf2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:12.587991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6ae7fe2b-c9f0-4c15-bb86-311e1305edf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:12.616917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:12.687380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:12.725899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a5d52-2f1c-4524-946a-0c0c90034141 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:12.727417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6e8a5d52-2f1c-4524-946a-0c0c90034141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:12.831618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a5d52-2f1c-4524-946a-0c0c90034141 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:12.859939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:12.905483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:12.906092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:12.906391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:12.906658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:12.908274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:13.014317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:13.015750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fba0d5-18f9-4e88-aeef-0b0cd1212299 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:13.033783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 721/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:15.077744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dbf7e-9456-4388-94dc-ba20cf4669d7 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:15.078314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dbf7e-9456-4388-94dc-ba20cf4669d7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:15.078314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dbf7e-9456-4388-94dc-ba20cf4669d7 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:15.079703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:15.085400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215220e-cd5d-48a8-8df1-1bd889a5419d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:15.085803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215220e-cd5d-48a8-8df1-1bd889a5419d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:15.085956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215220e-cd5d-48a8-8df1-1bd889a5419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:15.151912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0215220e-cd5d-48a8-8df1-1bd889a5419d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:15.153100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0215220e-cd5d-48a8-8df1-1bd889a5419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:15.162537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:15.252064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:15.290328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dbf7e-9456-4388-94dc-ba20cf4669d7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:15.291880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1d9dbf7e-9456-4388-94dc-ba20cf4669d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:15.393874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dbf7e-9456-4388-94dc-ba20cf4669d7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:15.448380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:15.466501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:15.467109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:15.467388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:15.467660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:15.469309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:15.577027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:15.578506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59524415-8ffb-4434-a90b-1e466b125217 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:15.597288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 739/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:17.763348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcde64-adfa-4758-b4a2-766e2cb90d8a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:17.763909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcde64-adfa-4758-b4a2-766e2cb90d8a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:17.763909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcde64-adfa-4758-b4a2-766e2cb90d8a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:17.765265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:17.771063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6a4a3-b617-4d99-a938-241db0a87d81 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:17.771063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6a4a3-b617-4d99-a938-241db0a87d81 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:17.771507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6a4a3-b617-4d99-a938-241db0a87d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:17.837403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b9b6a4a3-b617-4d99-a938-241db0a87d81 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:17.838594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b9b6a4a3-b617-4d99-a938-241db0a87d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:17.876983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:17.936913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:17.973983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcde64-adfa-4758-b4a2-766e2cb90d8a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:17.975493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-dcdcde64-adfa-4758-b4a2-766e2cb90d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:18.078039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcde64-adfa-4758-b4a2-766e2cb90d8a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:18.115720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:18.151605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:18.152214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:18.152489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:18.152756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:18.154391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:18.264302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:18.265775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a988b-dda4-4729-91fb-7f05c2ca9f51 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:18.284213 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 724/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:20.327828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab99f58-02f4-449c-8542-a33425266dc2 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:20.328379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab99f58-02f4-449c-8542-a33425266dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:20.328379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab99f58-02f4-449c-8542-a33425266dc2 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:20.329790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:20.335134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b906-0e1d-41e1-8d16-3b2424e7b5d1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:20.335570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b906-0e1d-41e1-8d16-3b2424e7b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:20.335666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b906-0e1d-41e1-8d16-3b2424e7b5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:20.400516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9263b906-0e1d-41e1-8d16-3b2424e7b5d1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:20.401739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9263b906-0e1d-41e1-8d16-3b2424e7b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:20.419246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:20.498270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:20.534409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab99f58-02f4-449c-8542-a33425266dc2 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:20.536608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3ab99f58-02f4-449c-8542-a33425266dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:20.762956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab99f58-02f4-449c-8542-a33425266dc2 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:20.828004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:20.835261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:20.853015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:20.853015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:20.853015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:20.853015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:20.948468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:20.950000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23496b04-4903-4b25-94f7-41744db79912 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:20.968809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 742/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:22.991084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d3551-e36d-4f18-baa4-0bcfac557567 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:22.991692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d3551-e36d-4f18-baa4-0bcfac557567 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:22.991692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d3551-e36d-4f18-baa4-0bcfac557567 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:22.993238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:22.998442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120fcd6-2f82-4fe6-9e62-cb2e33882c49 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:22.998892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120fcd6-2f82-4fe6-9e62-cb2e33882c49 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:22.998994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120fcd6-2f82-4fe6-9e62-cb2e33882c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:23.067676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b120fcd6-2f82-4fe6-9e62-cb2e33882c49 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:23.068893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b120fcd6-2f82-4fe6-9e62-cb2e33882c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:23.118376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:23.166679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:23.204914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d3551-e36d-4f18-baa4-0bcfac557567 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:23.206483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c80d3551-e36d-4f18-baa4-0bcfac557567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:23.308474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d3551-e36d-4f18-baa4-0bcfac557567 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:23.354716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:23.382704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:23.383316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:23.383590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:23.383856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:23.385487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:23.493288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:23.494721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07595e6d-4b6a-4a7d-a700-3576686fe7ff service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:23.512748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 727/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:33.164720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=22 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:33.510540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=66 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:43.151985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c4f6-f093-4696-95a4-344350907bff None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:43.152572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c4f6-f093-4696-95a4-344350907bff None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:43.152572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c4f6-f093-4696-95a4-344350907bff None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:43.154027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 744/1471] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:43.159135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89496d-5654-4d14-8cd1-6f9dec78624d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:43.159532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89496d-5654-4d14-8cd1-6f9dec78624d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:43.159681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89496d-5654-4d14-8cd1-6f9dec78624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:43.225668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-1b89496d-5654-4d14-8cd1-6f9dec78624d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:43.226871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1b89496d-5654-4d14-8cd1-6f9dec78624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:43.321698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 728/1472] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:43.404528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90ac3c-e945-410a-9a29-bba1de40b447 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:43.404757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90ac3c-e945-410a-9a29-bba1de40b447 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:43.405029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90ac3c-e945-410a-9a29-bba1de40b447 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:43.406430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 745/1473] 10.222.0.66 () {54 vars in 966 bytes} [Thu Aug 4 03:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:43.411486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e64a2-9853-4019-94d0-9040febf4786 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:43.411716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e64a2-9853-4019-94d0-9040febf4786 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:43.411985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e64a2-9853-4019-94d0-9040febf4786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:43.487441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-156e64a2-9853-4019-94d0-9040febf4786 None None] MFA Rules not processed for user `2e4a8d38d08c4619812d98a544510b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:43.488646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-156e64a2-9853-4019-94d0-9040febf4786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:43.509962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:43.566199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 729/1474] 10.222.0.66 () {58 vars in 1063 bytes} [Thu Aug 4 03:33:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:43.574542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90ac3c-e945-410a-9a29-bba1de40b447 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:43.576169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8b90ac3c-e945-410a-9a29-bba1de40b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:43.685634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90ac3c-e945-410a-9a29-bba1de40b447 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:43.761658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:43.762277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:43.762603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:43.762826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:43.764452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:43.821661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:43.876704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:43.878190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84277d8-7e13-44ad-aece-94daf0214930 service glance] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:43.896874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 746/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:45.992235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea29e8-e9e6-4f4c-815d-9a2e64281e15 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:45.993036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea29e8-e9e6-4f4c-815d-9a2e64281e15 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:45.993036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea29e8-e9e6-4f4c-815d-9a2e64281e15 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:45.994437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 730/1476] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:45.999661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399502c-704b-4eca-85be-b47f503e8383 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:46.000102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399502c-704b-4eca-85be-b47f503e8383 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:46.000212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399502c-704b-4eca-85be-b47f503e8383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:46.064972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-d399502c-704b-4eca-85be-b47f503e8383 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:46.066116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d399502c-704b-4eca-85be-b47f503e8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:46.161455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 747/1477] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:46.185336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea29e8-e9e6-4f4c-815d-9a2e64281e15 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:46.186855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f5ea29e8-e9e6-4f4c-815d-9a2e64281e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:46.313281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea29e8-e9e6-4f4c-815d-9a2e64281e15 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:46.362531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:46.406884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:46.407610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:46.407887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:46.408157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:46.408850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:46.410659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:33:46.410659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:33:46.410659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:33:46.411863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0a826-a70c-4696-bdd7-3fc8ea016e2e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:46.431923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 731/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Aug 4 03:33:46 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:33:48.447228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56241a8f-bc9f-4fd1-b074-df819953b6b6 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:48.447731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56241a8f-bc9f-4fd1-b074-df819953b6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:48.447731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56241a8f-bc9f-4fd1-b074-df819953b6b6 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:48.449039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 748/1479] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:48.454226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8803-2989-4961-b2be-d1528683dbe3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:48.454615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8803-2989-4961-b2be-d1528683dbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:48.454788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f8803-2989-4961-b2be-d1528683dbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:48.520688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5a1f8803-2989-4961-b2be-d1528683dbe3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:48.522127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5a1f8803-2989-4961-b2be-d1528683dbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:48.619553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 732/1480] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:48.663327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56241a8f-bc9f-4fd1-b074-df819953b6b6 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:48.664847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-56241a8f-bc9f-4fd1-b074-df819953b6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:48.674565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:48.809910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56241a8f-bc9f-4fd1-b074-df819953b6b6 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:48.890047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:48.923294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:48.923910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:48.924186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:48.924453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:48.926104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:49.015624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:49.017246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96455e90-0cf0-43fd-925a-6b3fcfd49f05 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:49.035537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 749/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:51.228431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e838-bc5b-4a1b-9bf2-ecfbf14cc857 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:51.228953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e838-bc5b-4a1b-9bf2-ecfbf14cc857 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:51.228953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1e838-bc5b-4a1b-9bf2-ecfbf14cc857 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:51.230295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 733/1482] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:51.235441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b456d-e54e-4bb1-a15f-ae8ccd7dd158 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:51.235869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b456d-e54e-4bb1-a15f-ae8ccd7dd158 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:51.236016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b456d-e54e-4bb1-a15f-ae8ccd7dd158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:51.303583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f00b456d-e54e-4bb1-a15f-ae8ccd7dd158 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:51.304775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f00b456d-e54e-4bb1-a15f-ae8ccd7dd158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:51.351009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:51.401542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 750/1483] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:33:51.439528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1e838-bc5b-4a1b-9bf2-ecfbf14cc857 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:51.441034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ebf1e838-bc5b-4a1b-9bf2-ecfbf14cc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:51.463766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:51.545357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1e838-bc5b-4a1b-9bf2-ecfbf14cc857 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:51.617807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:51.618424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:51.618707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:51.618983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:51.620599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:51.693258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:51.728131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:51.729613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be64828c-00fe-4908-a89e-31224d279e55 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:51.747543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 734/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:53.820724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03173-862d-4327-a67d-c9d25d43aa66 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:53.821289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03173-862d-4327-a67d-c9d25d43aa66 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:53.821400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf03173-862d-4327-a67d-c9d25d43aa66 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:53.822744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 751/1485] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:53.827901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266eae-538f-4b93-b07e-4a773fc7e71d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:53.828353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266eae-538f-4b93-b07e-4a773fc7e71d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:53.828524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266eae-538f-4b93-b07e-4a773fc7e71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:53.895620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e2266eae-538f-4b93-b07e-4a773fc7e71d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:53.896931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e2266eae-538f-4b93-b07e-4a773fc7e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:53.995133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 735/1486] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:33:54.033389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03173-862d-4327-a67d-c9d25d43aa66 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:54.034930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fbf03173-862d-4327-a67d-c9d25d43aa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:54.138912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf03173-862d-4327-a67d-c9d25d43aa66 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:54.212771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:54.214596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:54.215235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:54.215512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:54.215780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:54.217444 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:54.326391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:54.327838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febcc68-3bb0-4128-94d8-e9741d477280 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:54.346377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 752/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:56.422730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2e361-c674-4fab-ba58-48d265768847 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:56.423281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2e361-c674-4fab-ba58-48d265768847 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:56.423281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2e361-c674-4fab-ba58-48d265768847 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:56.424617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 736/1488] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:56.430499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2dfd2-8b97-42a0-9336-a1304a9b6d7a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:56.430916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2dfd2-8b97-42a0-9336-a1304a9b6d7a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:56.433444 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2dfd2-8b97-42a0-9336-a1304a9b6d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:56.496579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8dc2dfd2-8b97-42a0-9336-a1304a9b6d7a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:56.497797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8dc2dfd2-8b97-42a0-9336-a1304a9b6d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:56.548252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:56.592865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 753/1489] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:56.635587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2e361-c674-4fab-ba58-48d265768847 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:56.637113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-78b2e361-c674-4fab-ba58-48d265768847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:56.675303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:56.742917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2e361-c674-4fab-ba58-48d265768847 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:56.814070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:56.814679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:56.814957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:56.815227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:56.816840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:56.896633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:56.925579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:56.927221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c16e0-d769-4ff8-a6a4-da88ce013d78 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:56.945634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 737/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:33:59.356928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410808-ae4a-4d41-ab7b-ecaee949bb56 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:59.357537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410808-ae4a-4d41-ab7b-ecaee949bb56 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:59.357537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410808-ae4a-4d41-ab7b-ecaee949bb56 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:59.358913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 754/1491] 10.222.0.66 () {54 vars in 927 bytes} [Thu Aug 4 03:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:33:59.364561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e0f1b-2e8f-409b-b40d-7e663d4ffe04 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:59.365030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e0f1b-2e8f-409b-b40d-7e663d4ffe04 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:59.365343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e0f1b-2e8f-409b-b40d-7e663d4ffe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:59.431205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b08e0f1b-2e8f-409b-b40d-7e663d4ffe04 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:33:59.432336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b08e0f1b-2e8f-409b-b40d-7e663d4ffe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:59.530123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 738/1492] 10.222.0.66 () {58 vars in 1024 bytes} [Thu Aug 4 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:33:59.569239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410808-ae4a-4d41-ab7b-ecaee949bb56 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:33:59.570814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1a410808-ae4a-4d41-ab7b-ecaee949bb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:59.673688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410808-ae4a-4d41-ab7b-ecaee949bb56 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:33:59.745634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:33:59.746180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:33:59.746461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:33:59.746733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:33:59.748344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:33:59.757777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:33:59.856775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:33:59.858263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb3350-ac41-4418-93b3-305aa4ca3d23 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:33:59.876886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 755/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:34:09.528329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=82 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:34:09.874589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=50 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:38.667385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:38.667957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:38.667957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:38.744857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:35:38.746097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:38.841045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 739/1494] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:35:38.849281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:38.849782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:38.849890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:38.924692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:35:38.925830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:39.029020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 756/1495] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:35:39.033773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:39.035067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:39.046469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c343d08-f5a0-440d-8057-b428cc4c8080 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:39.107713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:39.140908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:39.141545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:39.141824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:39.142093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:39.142784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:35:39.143078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:39.144102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:39.144102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:35:39.144102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:35:39.145721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:39.157437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 740/1496] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:35:39.166013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:39.176360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c1faf-ba5a-4eba-8172-a7a686294408 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:39.239324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:39.274282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:39.274897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:39.275176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:39.275446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:39.276116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:39.277881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:39.277881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:35:39.277881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:35:39.279095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:39.385384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 757/1497] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:35:39 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:35:39.389789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:39.392628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:39.423314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4f0c-1b03-48e6-bdc4-62db4f5363ce admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:39.515628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:39.516163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:39.516437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:39.516704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:39.517405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:39.518414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:39.518414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:35:39.518414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:35:39.519726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:39.652946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 741/1498] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:35:39 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:35:39.657047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:39.658611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:39.720478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60205b82-3710-4e67-b214-5f96a8d9d97f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:39.803589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:39.813542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:35:39.820463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:35:39.821025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:39.830357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 758/1499] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:35:39.835403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:39.836946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:39.848950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae7d30-eb2e-4cdd-a240-b41a02e91ab1 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:39.875563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:39.946236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:39.946851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:39.947124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:39.947388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:39.948077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:39.949077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:39.949077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:35:39.949077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:35:39.949997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8a8ec-cc45-4781-98ec-85429d11fd4f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:39.962997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 742/1500] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:35:39.967195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:39.968702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:40.002835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e284b7-48e9-43af-a10c-c8b0bd9200d1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:40.096996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:40.097641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:40.098020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:40.098291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] PATH_INFO: `/v3/projects/08b91d4e9b1d436eb7dbe05627d34027/users/4615378a845b4869be196f23ae9e4d95/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:40.120232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b91d4e9b1d436eb7dbe05627d34027, user_id=4615378a845b4869be196f23ae9e4d95, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:40.125558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:40.125558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:35:40.125558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:35:40.125558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23a78b-ebb9-4fd9-9f24-3884f8d8770f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:40.224565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 759/1501] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:35:39 2022] PUT /identity/v3/projects/08b91d4e9b1d436eb7dbe05627d34027/users/4615378a845b4869be196f23ae9e4d95/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:35:40.236648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0ab1-a0a3-4778-be4c-a0d5a33068c8 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:40.236938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0ab1-a0a3-4778-be4c-a0d5a33068c8 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:40.237238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db0ab1-a0a3-4778-be4c-a0d5a33068c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:40.295238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c0db0ab1-a0a3-4778-be4c-a0d5a33068c8 None None] MFA Rules not processed for user `4615378a845b4869be196f23ae9e4d95`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:35:40.296446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c0db0ab1-a0a3-4778-be4c-a0d5a33068c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:40.373714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 743/1502] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:35:40.386928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4b8c-befc-46dd-bb2e-d67da9cc3f0a None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:40.387248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4b8c-befc-46dd-bb2e-d67da9cc3f0a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:40.387524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4b8c-befc-46dd-bb2e-d67da9cc3f0a None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:40.388870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 760/1503] 10.222.0.66 () {54 vars in 964 bytes} [Thu Aug 4 03:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:35:40.393596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ca63d-678e-4b71-a096-2060a3361827 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:40.393882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ca63d-678e-4b71-a096-2060a3361827 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:40.394156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ca63d-678e-4b71-a096-2060a3361827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:40.424791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:40.468738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9d7ca63d-678e-4b71-a096-2060a3361827 None None] MFA Rules not processed for user `428e53d5cb9e4a2996295b94e364ece4`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:35:40.469968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9d7ca63d-678e-4b71-a096-2060a3361827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:40.545104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 744/1504] 10.222.0.66 () {58 vars in 1061 bytes} [Thu Aug 4 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:35:40.553638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4b8c-befc-46dd-bb2e-d67da9cc3f0a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:40.555137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d4cd4b8c-befc-46dd-bb2e-d67da9cc3f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:40.603892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:40.657497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4b8c-befc-46dd-bb2e-d67da9cc3f0a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:40.730760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:40.731374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:40.731655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:40.731925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:40.733647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:40.830729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:40.832743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:40.836246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:40.850965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 761/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:35:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:35:40.895912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd295db-e60f-43ee-b810-f99a3e83cf6b None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:40.896371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd295db-e60f-43ee-b810-f99a3e83cf6b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:40.896477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd295db-e60f-43ee-b810-f99a3e83cf6b None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:40.897745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 745/1506] 10.222.0.66 () {52 vars in 830 bytes} [Thu Aug 4 03:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:35:40.901382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:40.902582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:41.006963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65264cd5-0094-401d-9263-14f7aade7840 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:41.100036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:41.100672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:41.100948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:41.101241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] PATH_INFO: `/v3/users/4615378a845b4869be196f23ae9e4d95` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:41.113131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=4615378a845b4869be196f23ae9e4d95)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:41.114469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:41.114469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:35:41.114469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:35:41.115690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:41.125500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:41.323495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4615378a845b4869be196f23ae9e4d95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:41.323859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4615378a845b4869be196f23ae9e4d95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:41.324408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4615378a845b4869be196f23ae9e4d95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:41.347067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4615378a845b4869be196f23ae9e4d95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:41.412039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4615378a845b4869be196f23ae9e4d95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:41.431713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 762/1507] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:35:40 2022] DELETE /identity/v3/users/4615378a845b4869be196f23ae9e4d95 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:35:41.436280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:41.436562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:41.436833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:41.493528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:35:41.494718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:41.569544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:41.581479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 746/1508] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:35:41.592130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:41.593389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:41.692647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d41f4-fb0b-4ad0-8d59-606b218a2bda admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:41.748370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:41.766816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:41.767434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:41.767712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:41.770927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:41.770927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:41.876871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:41.878360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5af3b-7e77-4128-94ff-abe36d31b268 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:41.897590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 763/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:35:42.369537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:35:42.370764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:35:42.383454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18723c2-3eae-464a-8be2-60c186a949ab None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:35:42.483359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:35:42.484000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:35:42.484282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:35:42.484557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] PATH_INFO: `/v3/projects/08b91d4e9b1d436eb7dbe05627d34027` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:35:42.494726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b91d4e9b1d436eb7dbe05627d34027)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:35:42.495755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:35:42.495755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:35:42.495755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:35:42.496974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:35:42.593895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b91d4e9b1d436eb7dbe05627d34027', 'request_id': 'req-a023b150-0260-4834-86f4-b117afc4de7c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:42.606803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b91d4e9b1d436eb7dbe05627d34027', 'request_id': 'req-a023b150-0260-4834-86f4-b117afc4de7c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:42.693035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b91d4e9b1d436eb7dbe05627d34027', 'request_id': 'req-a023b150-0260-4834-86f4-b117afc4de7c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:42.716605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] The token cache is being invalidate because project 08b91d4e9b1d436eb7dbe05627d34027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:35:42.716877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:42.717333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a023b150-0260-4834-86f4-b117afc4de7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:35:42.718748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 747/1510] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:35:42 2022] DELETE /identity/v3/projects/08b91d4e9b1d436eb7dbe05627d34027 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:35:51.895016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=62 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:35:52.714535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:37:09.383410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be3ff7-ca35-4c72-bac2-268f5adb5968 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:09.384003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be3ff7-ca35-4c72-bac2-268f5adb5968 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:09.384003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be3ff7-ca35-4c72-bac2-268f5adb5968 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:09.385362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:09.390276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328bbe8-4014-4e78-8b00-38affb95e8b2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:09.390696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328bbe8-4014-4e78-8b00-38affb95e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:09.390830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328bbe8-4014-4e78-8b00-38affb95e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:09.473048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4328bbe8-4014-4e78-8b00-38affb95e8b2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:09.475687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4328bbe8-4014-4e78-8b00-38affb95e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:09.579901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:09.734583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be3ff7-ca35-4c72-bac2-268f5adb5968 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:09.736692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-21be3ff7-ca35-4c72-bac2-268f5adb5968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:09.857531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be3ff7-ca35-4c72-bac2-268f5adb5968 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:09.863381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2bec7f-74f9-48fd-9a4e-3d10cb169837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:09.864087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bec7f-74f9-48fd-9a4e-3d10cb169837 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:09.864848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bec7f-74f9-48fd-9a4e-3d10cb169837 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:09.865499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2bec7f-74f9-48fd-9a4e-3d10cb169837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:09.873699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bec7f-74f9-48fd-9a4e-3d10cb169837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:09.875288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2bec7f-74f9-48fd-9a4e-3d10cb169837 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:09.885119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 765/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:10.716880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c3eba-2a2e-4a0f-87df-1bc3663cc900 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:10.717507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c3eba-2a2e-4a0f-87df-1bc3663cc900 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:10.717507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c3eba-2a2e-4a0f-87df-1bc3663cc900 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:10.719113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:10.723906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7a52-a916-444e-b43b-330d9faace2b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:10.724463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7a52-a916-444e-b43b-330d9faace2b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:10.724463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7a52-a916-444e-b43b-330d9faace2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:10.744538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-120c7a52-a916-444e-b43b-330d9faace2b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:10.745768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-120c7a52-a916-444e-b43b-330d9faace2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:10.757582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:10.916571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c3eba-2a2e-4a0f-87df-1bc3663cc900 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:10.932709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c3eba-2a2e-4a0f-87df-1bc3663cc900 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:10.938395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb61fc3-db61-4b37-a20d-d1def25dd7ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:10.939007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb61fc3-db61-4b37-a20d-d1def25dd7ac service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:10.939286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb61fc3-db61-4b37-a20d-d1def25dd7ac service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:10.939558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb61fc3-db61-4b37-a20d-d1def25dd7ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:10.948532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb61fc3-db61-4b37-a20d-d1def25dd7ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:10.950167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb61fc3-db61-4b37-a20d-d1def25dd7ac service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:10.960360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 750/1516] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:14.143403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392060-b318-4ab4-b1d9-2bc66a4e5dec None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:14.143950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392060-b318-4ab4-b1d9-2bc66a4e5dec None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:14.143950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392060-b318-4ab4-b1d9-2bc66a4e5dec None None] PATH_INFO: `/v3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:14.145279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 767/1517] 10.222.0.42 () {54 vars in 925 bytes} [Thu Aug 4 03:37:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:37:14.152044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a42d09-4fb5-492b-a5bb-ce55c4bcee38 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:14.152497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a42d09-4fb5-492b-a5bb-ce55c4bcee38 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:14.152607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a42d09-4fb5-492b-a5bb-ce55c4bcee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:14.213205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e6a42d09-4fb5-492b-a5bb-ce55c4bcee38 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:14.214438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e6a42d09-4fb5-492b-a5bb-ce55c4bcee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:14.231303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:37:14.289861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 751/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Aug 4 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:18.188183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba441c-1c33-4066-9817-3eb8b0de5836 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:18.188678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba441c-1c33-4066-9817-3eb8b0de5836 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:18.188774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba441c-1c33-4066-9817-3eb8b0de5836 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:18.190098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 768/1519] 10.222.0.66 () {54 vars in 961 bytes} [Thu Aug 4 03:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:18.196022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b63918-b053-4732-804b-5d6124412193 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:18.196474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b63918-b053-4732-804b-5d6124412193 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:18.196740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b63918-b053-4732-804b-5d6124412193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:18.215791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e8b63918-b053-4732-804b-5d6124412193 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:18.216977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e8b63918-b053-4732-804b-5d6124412193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:18.225486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 752/1520] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Aug 4 03:37:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:18.232820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba441c-1c33-4066-9817-3eb8b0de5836 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:18.247007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba441c-1c33-4066-9817-3eb8b0de5836 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:18.251920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02ed24-2780-46dc-be86-21db6731e993 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:18.252526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02ed24-2780-46dc-be86-21db6731e993 service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:18.252809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02ed24-2780-46dc-be86-21db6731e993 service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:18.253085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02ed24-2780-46dc-be86-21db6731e993 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:18.262019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02ed24-2780-46dc-be86-21db6731e993 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:18.263609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02ed24-2780-46dc-be86-21db6731e993 service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:18.273282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 769/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:37:22.187824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66325-1b49-4307-9025-832242f753ec None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:22.188266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66325-1b49-4307-9025-832242f753ec None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:22.188361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66325-1b49-4307-9025-832242f753ec None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:22.189760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:37:22.195258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32fb1-3196-44d7-a22f-fe845d49a265 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:22.196272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32fb1-3196-44d7-a22f-fe845d49a265 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:22.196272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a32fb1-3196-44d7-a22f-fe845d49a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:22.215662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-68a32fb1-3196-44d7-a22f-fe845d49a265 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:22.216862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-68a32fb1-3196-44d7-a22f-fe845d49a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:22.224801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:22.378187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66325-1b49-4307-9025-832242f753ec None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:22.393181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66325-1b49-4307-9025-832242f753ec None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:22.399252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f68960-1639-43cb-a08b-b1dd9dc8c983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:22.399851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f68960-1639-43cb-a08b-b1dd9dc8c983 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:22.400134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f68960-1639-43cb-a08b-b1dd9dc8c983 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:22.400409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f68960-1639-43cb-a08b-b1dd9dc8c983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:22.408437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f68960-1639-43cb-a08b-b1dd9dc8c983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:22.410063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f68960-1639-43cb-a08b-b1dd9dc8c983 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:22.419457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 754/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:23.245405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8597d-c9ea-4986-a299-8f2a58ba5c6d None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:23.246026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8597d-c9ea-4986-a299-8f2a58ba5c6d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:23.246026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8597d-c9ea-4986-a299-8f2a58ba5c6d None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:23.247362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 03:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:23.252196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2f4b-c9c8-480c-9af1-ef26c798d119 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:23.252651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2f4b-c9c8-480c-9af1-ef26c798d119 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:23.252794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff2f4b-c9c8-480c-9af1-ef26c798d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:23.273896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3cff2f4b-c9c8-480c-9af1-ef26c798d119 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:23.275083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3cff2f4b-c9c8-480c-9af1-ef26c798d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:23.282820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 03:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:23.434298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8597d-c9ea-4986-a299-8f2a58ba5c6d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:23.448562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8597d-c9ea-4986-a299-8f2a58ba5c6d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:23.454320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab95dc-b1e9-4285-8107-a591bd46fb19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:23.454925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab95dc-b1e9-4285-8107-a591bd46fb19 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:23.455208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab95dc-b1e9-4285-8107-a591bd46fb19 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:23.455482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab95dc-b1e9-4285-8107-a591bd46fb19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:23.464169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab95dc-b1e9-4285-8107-a591bd46fb19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:23.465812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab95dc-b1e9-4285-8107-a591bd46fb19 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:23.475158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 772/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:27.339243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6babfb-9531-4e79-9a60-7c8536a22d61 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:27.339767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6babfb-9531-4e79-9a60-7c8536a22d61 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:27.339767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6babfb-9531-4e79-9a60-7c8536a22d61 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:27.341166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:27.346223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23765c6b-f782-4b2b-86ce-b768887976b0 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:27.346603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23765c6b-f782-4b2b-86ce-b768887976b0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:27.346764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23765c6b-f782-4b2b-86ce-b768887976b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:27.366427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-23765c6b-f782-4b2b-86ce-b768887976b0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:27.368256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-23765c6b-f782-4b2b-86ce-b768887976b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:27.375894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:27.448248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6babfb-9531-4e79-9a60-7c8536a22d61 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:27.449952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-cd6babfb-9531-4e79-9a60-7c8536a22d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:27.533852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6babfb-9531-4e79-9a60-7c8536a22d61 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:27.537507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f1347-5d42-49ec-b31c-a9f00743a1d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:27.538121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f1347-5d42-49ec-b31c-a9f00743a1d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:27.538947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f1347-5d42-49ec-b31c-a9f00743a1d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:27.539222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f1347-5d42-49ec-b31c-a9f00743a1d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:27.548061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f1347-5d42-49ec-b31c-a9f00743a1d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:27.549548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f1347-5d42-49ec-b31c-a9f00743a1d7 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:27.559485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 757/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:29.582645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf72105-7b54-4e34-844e-83422e3a2190 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:29.583237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf72105-7b54-4e34-844e-83422e3a2190 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:29.583237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf72105-7b54-4e34-844e-83422e3a2190 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:29.584536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:29.590685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d931f3-bf97-4294-ac25-f59cfb0ecbb1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:29.591150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d931f3-bf97-4294-ac25-f59cfb0ecbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:29.591256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d931f3-bf97-4294-ac25-f59cfb0ecbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:29.611217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-74d931f3-bf97-4294-ac25-f59cfb0ecbb1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:29.612447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-74d931f3-bf97-4294-ac25-f59cfb0ecbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:29.620555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:29.656578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf72105-7b54-4e34-844e-83422e3a2190 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:29.667820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf72105-7b54-4e34-844e-83422e3a2190 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:29.671328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f63d0-3241-42b5-9810-e701833d48b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:29.671929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f63d0-3241-42b5-9810-e701833d48b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:29.672208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f63d0-3241-42b5-9810-e701833d48b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:29.672481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f63d0-3241-42b5-9810-e701833d48b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:29.681945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f63d0-3241-42b5-9810-e701833d48b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:29.683376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f63d0-3241-42b5-9810-e701833d48b0 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:29.692523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 775/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:33.193095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db0f51-b3e8-4b77-bec0-84acb8736f5d None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:33.193588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db0f51-b3e8-4b77-bec0-84acb8736f5d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:33.193701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db0f51-b3e8-4b77-bec0-84acb8736f5d None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:33.195803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:33.201004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bfda7-ae9b-4f20-8d91-ccffb94ea850 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:33.201454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bfda7-ae9b-4f20-8d91-ccffb94ea850 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:33.201597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bfda7-ae9b-4f20-8d91-ccffb94ea850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:33.209143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=81 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:37:33.220986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-344bfda7-ae9b-4f20-8d91-ccffb94ea850 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:33.230437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-344bfda7-ae9b-4f20-8d91-ccffb94ea850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:33.230437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:33.535404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db0f51-b3e8-4b77-bec0-84acb8736f5d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:33.537357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-12db0f51-b3e8-4b77-bec0-84acb8736f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:33.601502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:37:33.614811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db0f51-b3e8-4b77-bec0-84acb8736f5d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:33.618502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6baa0-2ba6-4363-919b-ee5e54a70cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:33.619108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6baa0-2ba6-4363-919b-ee5e54a70cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:33.619418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6baa0-2ba6-4363-919b-ee5e54a70cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:33.619707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6baa0-2ba6-4363-919b-ee5e54a70cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:33.628463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6baa0-2ba6-4363-919b-ee5e54a70cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:33.629936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6baa0-2ba6-4363-919b-ee5e54a70cb0 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:33.639966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 760/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:35.674291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5323e7e-8360-42a2-b283-43e5d9ccd79c None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:35.674865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5323e7e-8360-42a2-b283-43e5d9ccd79c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:35.674865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5323e7e-8360-42a2-b283-43e5d9ccd79c None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:35.676213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 03:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:37:35.681665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3f26-362e-4da9-8fa0-faeb9d33ca4e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:35.682103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3f26-362e-4da9-8fa0-faeb9d33ca4e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:35.682208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3f26-362e-4da9-8fa0-faeb9d33ca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:35.701959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-de3e3f26-362e-4da9-8fa0-faeb9d33ca4e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:37:35.703163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-de3e3f26-362e-4da9-8fa0-faeb9d33ca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:35.711992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 03:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:37:36.013381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5323e7e-8360-42a2-b283-43e5d9ccd79c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:37:36.015058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a5323e7e-8360-42a2-b283-43e5d9ccd79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:37:36.028578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5323e7e-8360-42a2-b283-43e5d9ccd79c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:37:36.032127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f378ec-4373-477d-9d93-3bdf2bdce570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:37:36.032731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f378ec-4373-477d-9d93-3bdf2bdce570 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:37:36.033015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f378ec-4373-477d-9d93-3bdf2bdce570 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:37:36.033330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f378ec-4373-477d-9d93-3bdf2bdce570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:37:36.042953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f378ec-4373-477d-9d93-3bdf2bdce570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:37:36.044396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f378ec-4373-477d-9d93-3bdf2bdce570 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:37:36.053764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 778/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:37:45.696962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=11 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:37:46.049670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=8 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:38:59.354670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8651620-107c-4ee3-a720-73611b1c1f34 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.355840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8651620-107c-4ee3-a720-73611b1c1f34 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.356361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8651620-107c-4ee3-a720-73611b1c1f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.382665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f8651620-107c-4ee3-a720-73611b1c1f34 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.384310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f8651620-107c-4ee3-a720-73611b1c1f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.400584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 762/1540] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.410076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af929e-bb75-441a-8b43-404ceefe1112 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.411100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af929e-bb75-441a-8b43-404ceefe1112 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.412296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af929e-bb75-441a-8b43-404ceefe1112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.427916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.428538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.429035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.442647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-22af929e-bb75-441a-8b43-404ceefe1112 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.444500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-22af929e-bb75-441a-8b43-404ceefe1112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.458852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 779/1541] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.460129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398ad46-7333-49c0-abd3-323f70a02a39 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.460418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398ad46-7333-49c0-abd3-323f70a02a39 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.460702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398ad46-7333-49c0-abd3-323f70a02a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.461141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.462597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.481317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 763/1542] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:38:59.482222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:38:59.495312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b398ad46-7333-49c0-abd3-323f70a02a39 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.496533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b398ad46-7333-49c0-abd3-323f70a02a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.498593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db197306-d8fa-4e0f-984b-c502f5f6116b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:38:59.507811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:38:59.508494 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.508776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.509058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.509781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:38:59.510080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:38:59.512770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 780/1543] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.513809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a74c7d-acaa-4b16-946e-ec399a59726d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:38:59.514062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.514352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.514626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.532886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 764/1544] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:38:59.534244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1c46b-bbe1-4442-a044-3b48ee64328d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.534535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1c46b-bbe1-4442-a044-3b48ee64328d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.534807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1c46b-bbe1-4442-a044-3b48ee64328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.548531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.549775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.567429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f4c1c46b-bbe1-4442-a044-3b48ee64328d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.568741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f4c1c46b-bbe1-4442-a044-3b48ee64328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.572832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 781/1545] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.573684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:38:59.586676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf385f-36d3-4233-87a2-31e3b6a2eddd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:38:59.591399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 765/1546] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.592669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.592954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.593235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.599913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6ab63-a2b7-47f7-89e4-72d106972455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:38:59.600537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6ab63-a2b7-47f7-89e4-72d106972455 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.600815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6ab63-a2b7-47f7-89e4-72d106972455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.601095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6ab63-a2b7-47f7-89e4-72d106972455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.601791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6ab63-a2b7-47f7-89e4-72d106972455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:38:59.603725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6ab63-a2b7-47f7-89e4-72d106972455 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:38:59.624406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.626204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.647334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 766/1547] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.648851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:38:59.665055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ee19-092d-4129-9fb7-a55f5d9110c4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:38:59.670876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:38:59.671509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.671789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.672063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.672767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:38:59.673067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:38:59.674726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:38:59.694339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 767/1548] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:38:59.694339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:38:59.697420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc71da9-5603-46f9-a8de-3880c3dbd8be admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:38:59.701976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:38:59.702544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.702824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.703099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.703773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:38:59.704074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:38:59.705701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a0f41-8d6d-415b-a0d6-a1c79a2ffcc3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:38:59.717289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 768/1549] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:38:59.718424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7bda9-d156-4710-a1da-e94645df1d12 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.718810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7bda9-d156-4710-a1da-e94645df1d12 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.719081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7bda9-d156-4710-a1da-e94645df1d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.739118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a6d7bda9-d156-4710-a1da-e94645df1d12 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.741080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a6d7bda9-d156-4710-a1da-e94645df1d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.749077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 769/1550] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.750263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134454e-fa77-4ec1-b83e-4f665fd85ca6 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.755957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134454e-fa77-4ec1-b83e-4f665fd85ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.755957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134454e-fa77-4ec1-b83e-4f665fd85ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.769381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-6134454e-fa77-4ec1-b83e-4f665fd85ca6 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.770554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6134454e-fa77-4ec1-b83e-4f665fd85ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.778570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 770/1551] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.779734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3324dba-fd7c-4f01-a4ff-295eac147fde None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.780017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3324dba-fd7c-4f01-a4ff-295eac147fde None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.780291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3324dba-fd7c-4f01-a4ff-295eac147fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.785102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 782/1552] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:38:59.785102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.785102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.785102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.803593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.803823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c3324dba-fd7c-4f01-a4ff-295eac147fde None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.804808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.804936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c3324dba-fd7c-4f01-a4ff-295eac147fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.864081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 771/1553] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.866302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 783/1554] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.866446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.866860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.867165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:38:59.867442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.883539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee63a08-7e57-4cab-8d3b-6f76459fa27f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:38:59.888375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2911ae-fc78-436d-8ea4-a3a1e919a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:38:59.889000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2911ae-fc78-436d-8ea4-a3a1e919a57b admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.889326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2911ae-fc78-436d-8ea4-a3a1e919a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.889604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2911ae-fc78-436d-8ea4-a3a1e919a57b admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.890273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2911ae-fc78-436d-8ea4-a3a1e919a57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:38:59.891149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:38:59.892214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2911ae-fc78-436d-8ea4-a3a1e919a57b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:38:59.892361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:38:59.900430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 772/1555] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:38:59.901409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:38:59.916397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefa733-ceec-412c-aefb-2b2b906cac54 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:38:59.921390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:38:59.922013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.922295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.922567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:38:59.923243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:38:59.924269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:38:59.924269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:38:59.924269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:38:59.925509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:38:59.982233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 784/1556] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/projects => generated 391 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:38:59.983485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ede64-1ee9-4114-8720-2752af1a9267 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:38:59.983779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ede64-1ee9-4114-8720-2752af1a9267 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:38:59.984050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ede64-1ee9-4114-8720-2752af1a9267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.003578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-124ede64-1ee9-4114-8720-2752af1a9267 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:00.007022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-124ede64-1ee9-4114-8720-2752af1a9267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:00.032618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 773/1557] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/projects => generated 403 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.033595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.066226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 785/1558] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:00.067942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.068235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.068510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.088263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:00.089495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:00.089647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:00.100568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17248fb2-27cf-4434-8669-fd36475bf38b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.107038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb27fd-e701-45bc-bbd6-a04117f367f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.107038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb27fd-e701-45bc-bbd6-a04117f367f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.107038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb27fd-e701-45bc-bbd6-a04117f367f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.107038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb27fd-e701-45bc-bbd6-a04117f367f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.107458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb27fd-e701-45bc-bbd6-a04117f367f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.108982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb27fd-e701-45bc-bbd6-a04117f367f4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.114068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:00.149397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 786/1559] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:00.150397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.159743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3188f30-c4bd-4eee-9972-e8a1e06dbc91 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.164745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.165388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.165671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.165944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.166684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:00.166984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.168004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:39:00.168004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:39:00.168004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:39:00.168913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbaecb8-b8de-4870-8795-40cd9ca57957 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.182252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 787/1560] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:00.184184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.184604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.185077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.189583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 774/1561] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.193328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.193328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.193328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.209020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:00.210398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:00.212641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:00.213879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:00.221336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 788/1562] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:00.221917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 775/1563] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:00.222519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.222738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.234596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654894fe-19ad-4d49-bc49-9a27b9cf847b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.234757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077f15a-64bb-4bcc-938b-aeeab41188b3 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.239905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.240066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.240581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.240864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.240957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.241139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.241277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.241492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.241852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.242146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.242871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:39:00.242871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:39:00.242871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:39:00.244058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.244157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.333896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 789/1564] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.334765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.344203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f1285-6c02-4f35-847f-63071d2d7d4e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.345649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 776/1565] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.346343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.352868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.353526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.353809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.354084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.354779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:00.355093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.356436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e516f2-6326-46c5-b341-f18355fc86f4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.356910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.361119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.361757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.362038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.362340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.363348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:00.363600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.366070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.371067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 790/1566] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:00.378599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 777/1567] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:00.382212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.382349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.392837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8d72-c10f-4bbc-bebe-9deeab688797 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.393008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5daa3-e7fd-467a-97ed-5decaee4b371 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.397575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.397729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.398174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.398285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.398435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.398579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.398719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.398841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.399391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.399499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.401000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.401373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.410879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 791/1568] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:00.414129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.424029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287562a-a85e-405b-a472-566e7be640a8 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.429456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.430057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.430343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.430622 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.431311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:00.431618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.433236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.445155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 792/1569] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:00.446034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.455333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2409c-2c7f-4a10-ba3a-fa4bcde38bdf admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.459899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.460467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.460748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.461021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.461758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:00.462058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.463634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.476030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 778/1570] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/projects => generated 363 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.476030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.479253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 793/1571] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:00.480071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.490565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ca326-de4f-47a2-b477-6bceccf65263 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.543468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec1289-8d63-41e8-864d-1f33dde2ade3 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.547277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.547889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.548019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.548279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.548560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.548662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.548879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.549150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.549733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.549872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.551331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.551472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.561822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 794/1572] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:00.565292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 779/1573] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:00.565429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.565541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.572852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d899bd5-1679-44e4-9627-e43499628c54 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.574490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeb8e9-a074-4ad6-a346-4710437f567d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.577569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.578165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.578515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.578728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.579023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.579392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.579632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.579907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.580184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.580839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.581346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.582802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.695996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 780/1574] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.696830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.706675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 795/1575] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.707704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.726721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:00.759344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:00.763124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9c93-2ecb-43e7-ac81-c301b1d6cbc5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.767787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.768413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.768692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.768967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.769705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.771352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.772310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced50c-b379-4a2c-ae62-dd766274389a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.776975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.777632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.777914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.778189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.778855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.780764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.781222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 781/1576] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:00.782081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.791790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6da608-00ad-4784-89b0-482bea27d121 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.796367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.796967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.797277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.797547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.798202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.800110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.866224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 796/1577] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.867169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.897352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 782/1578] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:00.897352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.904722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148e560-2d5d-4992-b096-9fc460f22f9a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.931944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb67b3-0957-4ca1-bcf0-3630f6e55eb6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.936864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.937515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.937795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.938068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.938732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.940686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.966340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.966959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.967243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.967522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.968222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.969884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:00.980181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 783/1579] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:00.980942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:00.991305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f49896-34f2-4f58-9156-d389e44f493f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:00.996092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:00.996683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:00.996967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:00.997272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:00.998025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:00.999601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.009923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 784/1580] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.010691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.019797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53529c37-440e-419a-bb07-44d3f657c7e4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.025000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.025771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.025921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.026155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.027074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.028702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.033001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 797/1581] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:00 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:01.039944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.096204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b385b8-5630-4c9b-8e7d-7644033de50d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.101951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.102556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.102835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.103111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.103761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.105726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.118620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 785/1582] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:01.119437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.128766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907f192-9a73-4562-8e03-15b05a4146b8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.133645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.134284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.134564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.134835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] PATH_INFO: `/v3/projects/8a97e25e7eae402f8f5c4222019ed87c/users/d9f4252210bd412ba24305b00a6e77b4/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.149050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a97e25e7eae402f8f5c4222019ed87c, user_id=d9f4252210bd412ba24305b00a6e77b4, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.150122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:39:01.150122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:39:01.150122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:39:01.152209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.200507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 798/1583] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:01.200507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.207879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68f7e9-9696-4426-8946-820bc717c89c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.212991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.213176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.213498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.213777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.214430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.216368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.236950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 786/1584] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:01 2022] PUT /identity/v3/projects/8a97e25e7eae402f8f5c4222019ed87c/users/d9f4252210bd412ba24305b00a6e77b4/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:01.238125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.294587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:01.299725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9ce76-f87b-473e-961b-8065443b8949 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.303598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 799/1585] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:01.303598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.307549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.307549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.307549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.307549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.307894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.308101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.313283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d58aa-69c4-4beb-b2d2-476c8b1a61a9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.318465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.319152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.319374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 787/1586] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.319504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.319847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] PATH_INFO: `/v3/projects/4272977c1f364aa3b63f3e0bbaddf2d9/users/26889216adb24ba3863a5d3a8c0a538d/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.320161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.329878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f3e56-9ebd-48bb-be8b-0f4040e8a4f0 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.339186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.339339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4272977c1f364aa3b63f3e0bbaddf2d9, user_id=26889216adb24ba3863a5d3a8c0a538d, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.339428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.339428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.339428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] PATH_INFO: `/v3/projects/7d9ff6952b014d0a9abb28dcf0b818b0/users/3ec8d131a9764028a096314a4ecd58f9/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.339690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.349052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:01.352253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9ff6952b014d0a9abb28dcf0b818b0, user_id=3ec8d131a9764028a096314a4ecd58f9, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.355013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.414056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 800/1587] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:01 2022] PUT /identity/v3/projects/4272977c1f364aa3b63f3e0bbaddf2d9/users/26889216adb24ba3863a5d3a8c0a538d/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:01.414880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.447330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 788/1588] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:01 2022] PUT /identity/v3/projects/7d9ff6952b014d0a9abb28dcf0b818b0/users/3ec8d131a9764028a096314a4ecd58f9/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:01.448183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.479337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c73ac-b66b-46d2-add3-778f5a29b57b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.509866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9d24a-cdf0-429e-bfcd-465c57c29258 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.514457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.515228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.515510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.515787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.516479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.518115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.527866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 789/1589] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.528661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.538417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6a000-03dc-4b53-94e0-7d24b7813d3d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.539285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9b076-6671-47c6-8772-0f6a1e43afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.539914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9b076-6671-47c6-8772-0f6a1e43afe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.540199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9b076-6671-47c6-8772-0f6a1e43afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.540472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9b076-6671-47c6-8772-0f6a1e43afe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.541183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9b076-6671-47c6-8772-0f6a1e43afe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.543019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b9812-bf78-407d-8710-73be1c3938d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.543157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9b076-6671-47c6-8772-0f6a1e43afe6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.543597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b9812-bf78-407d-8710-73be1c3938d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.543917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b9812-bf78-407d-8710-73be1c3938d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.544553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b9812-bf78-407d-8710-73be1c3938d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.545585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b9812-bf78-407d-8710-73be1c3938d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.547767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b9812-bf78-407d-8710-73be1c3938d2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.558752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 790/1590] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.563784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.563784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.563784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.579594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:01.580798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:01.588514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 791/1591] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:01.589304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.598465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d39775-b2e4-4c6f-a278-b046a9f24ed2 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.603407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.603995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.604274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.604545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.605251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.606973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.617649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 792/1592] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.619107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.626221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 801/1593] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:01.632385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.632385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.632385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.632666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2704e-77d3-4ebb-8528-37ef9dc7f564 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.634506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024effa-5f16-41f1-9270-f60980409318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.635111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024effa-5f16-41f1-9270-f60980409318 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.635434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024effa-5f16-41f1-9270-f60980409318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.635708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024effa-5f16-41f1-9270-f60980409318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.636407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024effa-5f16-41f1-9270-f60980409318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.638052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024effa-5f16-41f1-9270-f60980409318 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.648861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 793/1594] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.650290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.650644 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.650990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.658430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] MFA Rules not processed for user `26889216adb24ba3863a5d3a8c0a538d`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:01.659851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:01.670368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:01.671612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:01.680003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 794/1595] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:01.680834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.690095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76bcc6-0e11-449d-a36f-9b036ff6e5a5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.695411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.696011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.696288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.696587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.697295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.698903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.711976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 795/1596] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.711976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.718753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047d1e9-8941-4333-a279-7c6d57300db2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.723426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.724040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.724318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.724590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.725310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.726969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.731784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:01.735370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 802/1597] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:01.736230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.737136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 796/1598] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.737897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:01.738167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.749428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00bf08-113f-4e3d-9598-afc7d3d9226c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.754141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.754741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.755611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.756012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.756708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.758458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.772178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 797/1599] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.772178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.779535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c25de5-b063-426f-b8f8-47f60fc14ce0 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.784160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.784740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.785021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.785334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.786014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.788037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.795492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95764000-eb18-48b5-8ea1-6e7c3df2167d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.797836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 798/1600] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:01.798663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.801338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.802543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.802827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.803109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.810559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b487b9f-aef7-4ddc-9934-0e8a37db572d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.812449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.813938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.815349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.815988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.816273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.816547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] PATH_INFO: `/v3/projects/55a5665d4b124b4ea66bb2229c23e5c1/users/4050b9fcafc74194956129f056f971b0/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.825227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 803/1601] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:01.826392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.831775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55a5665d4b124b4ea66bb2229c23e5c1, user_id=4050b9fcafc74194956129f056f971b0, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.835201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.836615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501b489-7495-4693-acda-6e86e685c63a service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.841538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.842157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.842467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.842743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.852753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.854254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.863974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 804/1602] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:01.864804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.873630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef7539-9802-4ad6-9e79-c480e944739e service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.878299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.878939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.879222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.879496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] PATH_INFO: `/v3/projects/d3ace7ff4e72453bab404b46bb8b2cef/users/6daf35a4dd4249ecb83225047b942998/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.896718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ace7ff4e72453bab404b46bb8b2cef, user_id=6daf35a4dd4249ecb83225047b942998, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.899435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.911343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 799/1603] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:01 2022] PUT /identity/v3/projects/55a5665d4b124b4ea66bb2229c23e5c1/users/4050b9fcafc74194956129f056f971b0/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:01.912166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:01.946810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:01.977442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce9231-aa57-4c5e-b2c2-6a0c6e6bf357 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:01.982890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:01.983523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:01.983801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:01.984071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:01.984890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:01.986702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:01.997684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 805/1604] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:01 2022] PUT /identity/v3/projects/d3ace7ff4e72453bab404b46bb8b2cef/users/6daf35a4dd4249ecb83225047b942998/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:01.997684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.071377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2366-0522-4060-88b2-356c266bc4e5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.080249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.081501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.082013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.082483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] PATH_INFO: `/v3/projects/08ca5027abb74e11ac8c337217852767/users/3719507c161845789093a0c70189448f/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.088141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 800/1605] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:01 2022] POST /identity/v3/projects => generated 405 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:02.088141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.102467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ca5027abb74e11ac8c337217852767, user_id=3719507c161845789093a0c70189448f, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.105669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.157254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac202b0-59bd-4988-8c9a-e3514d550051 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.163143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be29717-be2b-4d28-88d6-a551e1f44d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.163819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29717-be2b-4d28-88d6-a551e1f44d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.164099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29717-be2b-4d28-88d6-a551e1f44d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.164367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be29717-be2b-4d28-88d6-a551e1f44d22 admin admin] PATH_INFO: `/v3/projects/f46b8dc64007429b805971d4c6f5b471/users/6dba436d0a67433f8c8ae7cd40d3f795/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.179915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be29717-be2b-4d28-88d6-a551e1f44d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46b8dc64007429b805971d4c6f5b471, user_id=6dba436d0a67433f8c8ae7cd40d3f795, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.182618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be29717-be2b-4d28-88d6-a551e1f44d22 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.202449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 806/1606] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:01 2022] PUT /identity/v3/projects/08ca5027abb74e11ac8c337217852767/users/3719507c161845789093a0c70189448f/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:02.203346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.266323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:02.280327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ca9d2-3181-4784-8c6a-0fcca92ad667 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.285663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d98ba-7c21-48af-bbee-5a8d3518bd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.286352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d98ba-7c21-48af-bbee-5a8d3518bd28 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.286581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d98ba-7c21-48af-bbee-5a8d3518bd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.287750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d98ba-7c21-48af-bbee-5a8d3518bd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.288624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d98ba-7c21-48af-bbee-5a8d3518bd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.294752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d98ba-7c21-48af-bbee-5a8d3518bd28 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.304978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 807/1607] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:02.306319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.306608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.306882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.318640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 801/1608] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:02 2022] PUT /identity/v3/projects/f46b8dc64007429b805971d4c6f5b471/users/6dba436d0a67433f8c8ae7cd40d3f795/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:02.320407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f057-42c3-47a9-aac6-6ef3c3074f6c None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.320846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f057-42c3-47a9-aac6-6ef3c3074f6c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.321303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f057-42c3-47a9-aac6-6ef3c3074f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.345694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] MFA Rules not processed for user `4050b9fcafc74194956129f056f971b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:02.345694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:02.354715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-bca3f057-42c3-47a9-aac6-6ef3c3074f6c None None] MFA Rules not processed for user `6daf35a4dd4249ecb83225047b942998`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:02.356081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bca3f057-42c3-47a9-aac6-6ef3c3074f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:02.417014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 808/1609] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:02.418210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.432675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 802/1610] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:02.434089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442f61f-64c9-4465-b100-e95424564326 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.434476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442f61f-64c9-4465-b100-e95424564326 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.434849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442f61f-64c9-4465-b100-e95424564326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.466781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-1442f61f-64c9-4465-b100-e95424564326 None None] MFA Rules not processed for user `3719507c161845789093a0c70189448f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:02.470120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1442f61f-64c9-4465-b100-e95424564326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:02.488967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:02.494868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6fb8e-f8af-4130-a131-2ec0cd0f1b2c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.499996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe09f7-57cd-4a6f-803a-16687518261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.500767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe09f7-57cd-4a6f-803a-16687518261e admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.501115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe09f7-57cd-4a6f-803a-16687518261e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.501349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe09f7-57cd-4a6f-803a-16687518261e admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.502012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe09f7-57cd-4a6f-803a-16687518261e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.504661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe09f7-57cd-4a6f-803a-16687518261e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.539721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 803/1611] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:02.540645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442f61f-64c9-4465-b100-e95424564326 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.551296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442f61f-64c9-4465-b100-e95424564326 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.557481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f70e23d-8320-48f2-ad32-5002f43a369e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.558259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e23d-8320-48f2-ad32-5002f43a369e admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.558610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e23d-8320-48f2-ad32-5002f43a369e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.561790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e23d-8320-48f2-ad32-5002f43a369e admin admin] PATH_INFO: `/v3/projects/1f44dc3db02e4cadbdb11bf0c87d1a4d/users/7485417b59cd4b2184f52d99435728ee/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.577247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70e23d-8320-48f2-ad32-5002f43a369e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f44dc3db02e4cadbdb11bf0c87d1a4d, user_id=7485417b59cd4b2184f52d99435728ee, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.590513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f70e23d-8320-48f2-ad32-5002f43a369e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.611301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 809/1612] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/users => generated 309 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:02.611500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.611698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.611971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.636455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:02.638228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:02.653037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 810/1613] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:02.654485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.667086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22bd9a-e766-4192-8f50-681e3eb79b2e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.675375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.676086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.676435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.677601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.678620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.680739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.701773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 804/1614] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:02 2022] PUT /identity/v3/projects/1f44dc3db02e4cadbdb11bf0c87d1a4d/users/7485417b59cd4b2184f52d99435728ee/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:02.713374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723afb6d-0e5f-456f-a600-e3b232603639 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.713374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723afb6d-0e5f-456f-a600-e3b232603639 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.713374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723afb6d-0e5f-456f-a600-e3b232603639 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.713374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 805/1615] 10.222.0.66 () {54 vars in 962 bytes} [Thu Aug 4 03:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:02.713374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57adc-824b-468f-9f7e-29ea89b34a40 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.713374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57adc-824b-468f-9f7e-29ea89b34a40 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.713833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f57adc-824b-468f-9f7e-29ea89b34a40 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.713833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 806/1616] 10.222.0.66 () {54 vars in 962 bytes} [Thu Aug 4 03:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:02.713833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f57adc-824b-468f-9f7e-29ea89b34a40 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.781078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f57adc-824b-468f-9f7e-29ea89b34a40 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.786385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd327b9-795c-43dc-96dc-520c00468e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.787184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd327b9-795c-43dc-96dc-520c00468e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.787647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd327b9-795c-43dc-96dc-520c00468e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.788117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd327b9-795c-43dc-96dc-520c00468e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.788833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd327b9-795c-43dc-96dc-520c00468e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.790847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd327b9-795c-43dc-96dc-520c00468e70 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.791349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 811/1617] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:02.792605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:02.806479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 807/1618] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:02.808336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150cc5f-28db-4e82-ba68-94ebb894a733 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.808630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150cc5f-28db-4e82-ba68-94ebb894a733 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.808910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150cc5f-28db-4e82-ba68-94ebb894a733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.846140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:02.858790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4150cc5f-28db-4e82-ba68-94ebb894a733 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:02.861299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4150cc5f-28db-4e82-ba68-94ebb894a733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:02.870690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b419-9a00-4509-92f6-e3ee70017401 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:02.874929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3cbec-1e14-468b-b77e-c89d0341091e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:02.876065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3cbec-1e14-468b-b77e-c89d0341091e service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.876487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3cbec-1e14-468b-b77e-c89d0341091e service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.877346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3cbec-1e14-468b-b77e-c89d0341091e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.956854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3cbec-1e14-468b-b77e-c89d0341091e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:02.957832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 808/1619] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:02.958906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3cbec-1e14-468b-b77e-c89d0341091e service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:02.960148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.960528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.960895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:02.970804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 812/1620] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:02.972298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:02.974923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:02.975432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.003353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] MFA Rules not processed for user `7485417b59cd4b2184f52d99435728ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:03.004785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:03.007971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:03.011455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:03.025545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 813/1621] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:03.026843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.039279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928cc08-922e-4d81-af36-c253bae911cf None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.044363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.045263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.045650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.045985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.046840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.048999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.072390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:03.092963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 809/1622] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:03.093753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.106775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324a0d0-8aec-4cc8-b960-3114f61cbb86 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.111798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.112486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.113034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.113564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] PATH_INFO: `/v3/projects/775ac8187f954e518a316e8b1ced6085/users/b921f5f0a20b4cf384f77d485fcb0fc2/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.133980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775ac8187f954e518a316e8b1ced6085, user_id=b921f5f0a20b4cf384f77d485fcb0fc2, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.136760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.141596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 814/1623] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:03 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:03.145592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.165605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2af83-97c4-4b08-b1ac-4b47d04c8975 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.182838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.183478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.183764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.184280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.241508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 810/1624] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:03 2022] PUT /identity/v3/projects/775ac8187f954e518a316e8b1ced6085/users/b921f5f0a20b4cf384f77d485fcb0fc2/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:03.242556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.252784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.254537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.264716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 815/1625] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:03.265721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.315799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65cf8a-4a09-4d29-874d-0f78801ed3f4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.328720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.329448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.332594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.332594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.337031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8b6d9-b2b5-4e0e-b408-a465397c3902 service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.398828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.399614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.400132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.400553 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.423939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.429304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.434642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:03.454082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 811/1626] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:03.455067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.520447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.524812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.556467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 816/1627] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:03.557228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.585698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:03.607611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322707-310a-438a-8c32-dfef366202ff service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.620738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1c320-3658-4b08-88a6-a2e19ff7b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.621073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1c320-3658-4b08-88a6-a2e19ff7b8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.622042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1c320-3658-4b08-88a6-a2e19ff7b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.622420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1c320-3658-4b08-88a6-a2e19ff7b8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.623482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1c320-3658-4b08-88a6-a2e19ff7b8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.627704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1c320-3658-4b08-88a6-a2e19ff7b8d7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.646187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 812/1628] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:03.647755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be293a81-8df2-4a42-b042-dcc1e023c4c0 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.648093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be293a81-8df2-4a42-b042-dcc1e023c4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.649335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be293a81-8df2-4a42-b042-dcc1e023c4c0 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.650658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 813/1629] 10.222.0.66 () {54 vars in 912 bytes} [Thu Aug 4 03:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:03.652405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be293a81-8df2-4a42-b042-dcc1e023c4c0 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.679054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92998fbe-c023-42dd-8052-882b6ce4b7e3 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.683992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be293a81-8df2-4a42-b042-dcc1e023c4c0 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.685178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca2fcc-0b34-408c-a2b7-ada296b38d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.686059 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca2fcc-0b34-408c-a2b7-ada296b38d1b admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.686845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca2fcc-0b34-408c-a2b7-ada296b38d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.687174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca2fcc-0b34-408c-a2b7-ada296b38d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.688351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca2fcc-0b34-408c-a2b7-ada296b38d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.691016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca2fcc-0b34-408c-a2b7-ada296b38d1b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.696427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.697160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.697468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.697740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] PATH_INFO: `/v3/projects/8493eb7dc7444855818eaefc8282cf23/users/7a8c51f01775455cb7570b761d947fab/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.717236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 817/1630] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:03.719420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67073-3309-4130-847c-01f191900ffe None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.721427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67073-3309-4130-847c-01f191900ffe None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.721719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d67073-3309-4130-847c-01f191900ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.732407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8493eb7dc7444855818eaefc8282cf23, user_id=7a8c51f01775455cb7570b761d947fab, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.738542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.756329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-93d67073-3309-4130-847c-01f191900ffe None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:03.758132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-93d67073-3309-4130-847c-01f191900ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:03.772185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 818/1631] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:39:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:03.773175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67073-3309-4130-847c-01f191900ffe None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.796925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d67073-3309-4130-847c-01f191900ffe None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.806002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bad927-7009-4575-9444-641e06c7d14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:03.807191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad927-7009-4575-9444-641e06c7d14e admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.815136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad927-7009-4575-9444-641e06c7d14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.819283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 814/1632] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:03 2022] PUT /identity/v3/projects/8493eb7dc7444855818eaefc8282cf23/users/7a8c51f01775455cb7570b761d947fab/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:03.820192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.820759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad927-7009-4575-9444-641e06c7d14e admin admin] PATH_INFO: `/v3/projects/775ac8187f954e518a316e8b1ced6085/users/b921f5f0a20b4cf384f77d485fcb0fc2/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.828688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bad927-7009-4575-9444-641e06c7d14e admin admin] RBAC: Authorizing `identity:create_grant(project_id=775ac8187f954e518a316e8b1ced6085, user_id=b921f5f0a20b4cf384f77d485fcb0fc2, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:03.835488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bad927-7009-4575-9444-641e06c7d14e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:03.878497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 819/1633] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:03 2022] PUT /identity/v3/projects/775ac8187f954e518a316e8b1ced6085/users/b921f5f0a20b4cf384f77d485fcb0fc2/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:03.880336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0e55f-36bf-4d6c-923b-7b13883600a1 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:03.880642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0e55f-36bf-4d6c-923b-7b13883600a1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:03.880919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0e55f-36bf-4d6c-923b-7b13883600a1 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:03.882729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 820/1634] 10.222.0.66 () {54 vars in 912 bytes} [Thu Aug 4 03:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:03.883516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0e55f-36bf-4d6c-923b-7b13883600a1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:03.973371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033684e7-132d-4325-ae51-d73c5e72e8e8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:03.988156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:04.023707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0e55f-36bf-4d6c-923b-7b13883600a1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:04.034758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0b438-2f02-4fb2-87e8-af32d2257d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:04.035406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0b438-2f02-4fb2-87e8-af32d2257d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:04.035727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0b438-2f02-4fb2-87e8-af32d2257d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:04.035968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0b438-2f02-4fb2-87e8-af32d2257d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:04.037276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0b438-2f02-4fb2-87e8-af32d2257d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:04.039032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0b438-2f02-4fb2-87e8-af32d2257d73 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:04.044359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e1b887-b0b0-4d44-82d5-e4c20d65b62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:04.045359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1b887-b0b0-4d44-82d5-e4c20d65b62f service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:04.045868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1b887-b0b0-4d44-82d5-e4c20d65b62f service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:04.046351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e1b887-b0b0-4d44-82d5-e4c20d65b62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:04.070140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 821/1635] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:04.074099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:04.075425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:04.077912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:04.140077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] MFA Rules not processed for user `b921f5f0a20b4cf384f77d485fcb0fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:04.148135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:04.157931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:04.211864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1b887-b0b0-4d44-82d5-e4c20d65b62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:04.211864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e1b887-b0b0-4d44-82d5-e4c20d65b62f service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:04.218615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 815/1636] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:04.220259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916ed8-4b23-471b-a458-116056810802 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:04.220967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916ed8-4b23-471b-a458-116056810802 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:04.221498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2916ed8-4b23-471b-a458-116056810802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:04.262044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e2916ed8-4b23-471b-a458-116056810802 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:04.263374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e2916ed8-4b23-471b-a458-116056810802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:04.267398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 822/1637] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:04 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:04.271335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:04.278892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 816/1638] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:39:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:04.290587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103bd8b-5f24-4bfa-ac9d-1469b5194933 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:04.297002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:04.303523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:04.303523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:04.303523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] PATH_INFO: `/v3/projects/8493eb7dc7444855818eaefc8282cf23/users/7a8c51f01775455cb7570b761d947fab/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:04.303523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8493eb7dc7444855818eaefc8282cf23, user_id=7a8c51f01775455cb7570b761d947fab, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:04.304904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:04.319621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 823/1639] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:04 2022] PUT /identity/v3/projects/8493eb7dc7444855818eaefc8282cf23/users/7a8c51f01775455cb7570b761d947fab/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:04.329488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:04.334184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:04.334184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:04.397680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] MFA Rules not processed for user `7a8c51f01775455cb7570b761d947fab`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:04.403863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:04.541293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 817/1640] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:04.948902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:05.058447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b77318-0caa-4e56-a884-99bd8b2d5910 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:05.068620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:05.071296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:05.071814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:05.072446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:05.191603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:05.196817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:05.211504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 824/1641] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:04 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:05.330377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:05.338404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:05.344587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:05.346217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa495810-1f88-466b-8225-99adea0aa591 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:05.349440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25752475-60f9-4508-9290-132baf88f16f service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:05.355309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7970e9-f76c-4ab2-a27a-1c811695e213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:05.356502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7970e9-f76c-4ab2-a27a-1c811695e213 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:05.356502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7970e9-f76c-4ab2-a27a-1c811695e213 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:05.361086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7970e9-f76c-4ab2-a27a-1c811695e213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:05.365974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:05.366889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:05.368154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:05.368584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:05.391324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:05.402916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:05.424186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 825/1642] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:05.523525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7970e9-f76c-4ab2-a27a-1c811695e213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:05.525018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7970e9-f76c-4ab2-a27a-1c811695e213 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:05.565545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 818/1643] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:06.507950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:06.866819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2519eaa-7e54-4076-932e-de95928f3abd service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:06.866819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:06.866819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:06.866819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:06.870291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] PATH_INFO: `/v3/users/26889216adb24ba3863a5d3a8c0a538d` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:06.870291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26889216adb24ba3863a5d3a8c0a538d)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:06.870291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:07.067061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26889216adb24ba3863a5d3a8c0a538d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:07.069505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26889216adb24ba3863a5d3a8c0a538d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:07.070841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26889216adb24ba3863a5d3a8c0a538d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:07.095521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26889216adb24ba3863a5d3a8c0a538d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:07.172612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26889216adb24ba3863a5d3a8c0a538d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:07.200162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 826/1644] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:06 2022] DELETE /identity/v3/users/26889216adb24ba3863a5d3a8c0a538d => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:07.202017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:07.202478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:07.207234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:07.272191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:07.272191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:07.359723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:07.392863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:07.396555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:07.396980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 819/1645] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:07.553994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f24563-bc72-4edd-bd9a-e8252a0cd698 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:07.565311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9d237-3bfa-4c14-9f72-48a83206a6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:07.566022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d237-3bfa-4c14-9f72-48a83206a6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:07.566380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d237-3bfa-4c14-9f72-48a83206a6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:07.566571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9d237-3bfa-4c14-9f72-48a83206a6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:07.581950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9d237-3bfa-4c14-9f72-48a83206a6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:07.590251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9d237-3bfa-4c14-9f72-48a83206a6e9 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:07.628187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 827/1646] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:39:07.832621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:07.834753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:07.853600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e9d16-420f-4768-806e-9f25a477cebb None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:07.878109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:07.879234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:07.880412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:07.880888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] PATH_INFO: `/v3/users/4050b9fcafc74194956129f056f971b0` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:07.882665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4050b9fcafc74194956129f056f971b0)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:07.883882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: WARNING py.warnings [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:39:07.883882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: warnings.warn(msg) Aug 04 03:39:07.883882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: Aug 04 03:39:07.885456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:08.335858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4050b9fcafc74194956129f056f971b0'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:08.373108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4050b9fcafc74194956129f056f971b0'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:08.375880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4050b9fcafc74194956129f056f971b0'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:08.437829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4050b9fcafc74194956129f056f971b0'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:08.573893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4050b9fcafc74194956129f056f971b0'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:08.581864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 820/1647] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:07 2022] DELETE /identity/v3/users/4050b9fcafc74194956129f056f971b0 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:08.590011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:08.590011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:08.590011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:08.638342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:08.640935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:08.676564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:08.829126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 828/1648] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:08.875000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:08.877350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:09.050140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a13a3-06ee-4689-bf7e-d284d1963ef6 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:09.065925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:09.070978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:09.071268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:09.076174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:09.110883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:09.112384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:09.156344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 821/1649] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:09.501221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:09.503088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:09.530507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587f2a7-87a0-4656-997b-fdaeda1dfdac None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:09.542742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:09.544074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:09.545296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:09.546611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] PATH_INFO: `/v3/projects/4272977c1f364aa3b63f3e0bbaddf2d9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:09.548882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4272977c1f364aa3b63f3e0bbaddf2d9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:09.551142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: WARNING py.warnings [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 03:39:09.551142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: warnings.warn(msg) Aug 04 03:39:09.551142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: Aug 04 03:39:09.554810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:09.769509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4272977c1f364aa3b63f3e0bbaddf2d9', 'request_id': 'req-da95ea5b-23f7-4389-bf53-e182938b7433'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:09.792978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4272977c1f364aa3b63f3e0bbaddf2d9', 'request_id': 'req-da95ea5b-23f7-4389-bf53-e182938b7433'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:09.868911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4272977c1f364aa3b63f3e0bbaddf2d9', 'request_id': 'req-da95ea5b-23f7-4389-bf53-e182938b7433'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:09.905223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] The token cache is being invalidate because project 4272977c1f364aa3b63f3e0bbaddf2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:09.905800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:09.907721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:09.912026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 829/1650] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:09 2022] DELETE /identity/v3/projects/4272977c1f364aa3b63f3e0bbaddf2d9 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:09.917338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:09.954800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e89df4-d3f9-40cd-9ade-47fd0b958c67 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:10.065283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:10.066296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:10.067015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:10.068137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] PATH_INFO: `/v3/users/b921f5f0a20b4cf384f77d485fcb0fc2` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:10.069613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b921f5f0a20b4cf384f77d485fcb0fc2)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:10.071841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:10.077580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:10.219832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:10.221837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:10.244891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95ea5b-23f7-4389-bf53-e182938b7433 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:10.266010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:10.267514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:10.268028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:10.269344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] PATH_INFO: `/v3/projects/55a5665d4b124b4ea66bb2229c23e5c1` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:10.271567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55a5665d4b124b4ea66bb2229c23e5c1)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:10.273795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:10.317594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b921f5f0a20b4cf384f77d485fcb0fc2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.325719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b921f5f0a20b4cf384f77d485fcb0fc2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.326802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b921f5f0a20b4cf384f77d485fcb0fc2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.354874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b921f5f0a20b4cf384f77d485fcb0fc2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.460256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b921f5f0a20b4cf384f77d485fcb0fc2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.472269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 822/1651] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:09 2022] DELETE /identity/v3/users/b921f5f0a20b4cf384f77d485fcb0fc2 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:10.486625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55a5665d4b124b4ea66bb2229c23e5c1', 'request_id': 'req-583b10a9-e6c3-406a-8d81-4361495833d7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.518092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55a5665d4b124b4ea66bb2229c23e5c1', 'request_id': 'req-583b10a9-e6c3-406a-8d81-4361495833d7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.612146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55a5665d4b124b4ea66bb2229c23e5c1', 'request_id': 'req-583b10a9-e6c3-406a-8d81-4361495833d7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.636211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] The token cache is being invalidate because project 55a5665d4b124b4ea66bb2229c23e5c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:10.639154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.640688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:10.644655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 830/1652] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:10 2022] DELETE /identity/v3/projects/55a5665d4b124b4ea66bb2229c23e5c1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:10.648962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:10.651580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:10.844948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c7818-82ad-4820-96a5-98851e079402 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:10.852709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:10.853609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:10.853898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:10.854647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] PATH_INFO: `/v3/users/7a8c51f01775455cb7570b761d947fab` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:10.855550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8c51f01775455cb7570b761d947fab)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:10.857948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:11.134974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8c51f01775455cb7570b761d947fab'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:11.144451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8c51f01775455cb7570b761d947fab'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:11.145620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8c51f01775455cb7570b761d947fab'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:11.178378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8c51f01775455cb7570b761d947fab'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:11.282776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8c51f01775455cb7570b761d947fab'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:11.298782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 823/1653] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:10 2022] DELETE /identity/v3/users/7a8c51f01775455cb7570b761d947fab => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:11.740315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:12.212720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:12.219024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:12.277601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:12.390432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b10a9-e6c3-406a-8d81-4361495833d7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:12.486273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:12.487182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:12.487888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:12.488311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] PATH_INFO: `/v3/projects/775ac8187f954e518a316e8b1ced6085` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:12.490265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=775ac8187f954e518a316e8b1ced6085)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:12.492380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:12.564152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:12.566137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:12.588847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79024a-d6bd-42d2-a8e7-e3f39c30ed00 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:12.604103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:12.605313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:12.605748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:12.606162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] PATH_INFO: `/v3/projects/8493eb7dc7444855818eaefc8282cf23` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:12.607861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8493eb7dc7444855818eaefc8282cf23)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:12.612991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:12.683700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775ac8187f954e518a316e8b1ced6085', 'request_id': 'req-036fdc46-467c-4231-9584-2f2f00ffde93'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.735038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775ac8187f954e518a316e8b1ced6085', 'request_id': 'req-036fdc46-467c-4231-9584-2f2f00ffde93'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.768153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8493eb7dc7444855818eaefc8282cf23', 'request_id': 'req-474ec224-ac30-43be-a3f1-404310658896'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.794948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8493eb7dc7444855818eaefc8282cf23', 'request_id': 'req-474ec224-ac30-43be-a3f1-404310658896'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.836863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775ac8187f954e518a316e8b1ced6085', 'request_id': 'req-036fdc46-467c-4231-9584-2f2f00ffde93'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.851865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] The token cache is being invalidate because project 775ac8187f954e518a316e8b1ced6085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:12.852039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.855308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-036fdc46-467c-4231-9584-2f2f00ffde93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:12.859405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 831/1654] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:12 2022] DELETE /identity/v3/projects/775ac8187f954e518a316e8b1ced6085 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:12.863674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323dc1-f013-4b94-92be-2a204c1d2a63 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:12.863967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323dc1-f013-4b94-92be-2a204c1d2a63 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:12.864277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323dc1-f013-4b94-92be-2a204c1d2a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:12.920634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a8323dc1-f013-4b94-92be-2a204c1d2a63 None None] MFA Rules not processed for user `d9f4252210bd412ba24305b00a6e77b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:12.922294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a8323dc1-f013-4b94-92be-2a204c1d2a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:12.991620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8493eb7dc7444855818eaefc8282cf23', 'request_id': 'req-474ec224-ac30-43be-a3f1-404310658896'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:13.003887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] The token cache is being invalidate because project 8493eb7dc7444855818eaefc8282cf23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:13.004179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:13.005619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-474ec224-ac30-43be-a3f1-404310658896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:13.011022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 824/1655] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:12 2022] DELETE /identity/v3/projects/8493eb7dc7444855818eaefc8282cf23 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:13.012657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b223-0652-4975-90b9-c7d6fe2492c4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.013094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b223-0652-4975-90b9-c7d6fe2492c4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.013552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875b223-0652-4975-90b9-c7d6fe2492c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.069438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 832/1656] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:13.070936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d43b6a-a644-4e7f-9739-a1f11bd85439 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.071384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d43b6a-a644-4e7f-9739-a1f11bd85439 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.071673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d43b6a-a644-4e7f-9739-a1f11bd85439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.095787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-78d43b6a-a644-4e7f-9739-a1f11bd85439 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:13.095921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f875b223-0652-4975-90b9-c7d6fe2492c4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:13.097188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-78d43b6a-a644-4e7f-9739-a1f11bd85439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:13.099787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f875b223-0652-4975-90b9-c7d6fe2492c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:13.203873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 833/1657] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:13.208780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e238fe-8c2c-4f43-be99-3d983e0cdfae None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.209671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e238fe-8c2c-4f43-be99-3d983e0cdfae None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.212222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e238fe-8c2c-4f43-be99-3d983e0cdfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.227614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 825/1658] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:13.231755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.232176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.232556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.242413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-45e238fe-8c2c-4f43-be99-3d983e0cdfae None None] MFA Rules not processed for user `d9f4252210bd412ba24305b00a6e77b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:13.244199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-45e238fe-8c2c-4f43-be99-3d983e0cdfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:13.255937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 834/1659] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:13.258395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.258828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.259529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.263739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:13.266118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:13.281295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 826/1660] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:13.282216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.285806 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:13.294610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:13.296434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:13.319980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 835/1661] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:13.321532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.335799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30f483-9f49-48cb-a551-1e523388f55d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.348244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.351313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.353309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.353309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.356454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:13.356932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.361249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.406056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 836/1662] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:13.407166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.433085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64dd9e-0337-4841-a313-38c5d054117c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.445655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.446401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.446691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.446969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.448190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:13.448190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.448969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa721f-a010-4ad9-b6e2-df36022ea25a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.449741 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.461313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.462556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.463021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.463420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.469177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 837/1663] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:13.470183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.474959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.477106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.488658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b13324-e0da-45b7-8620-0f81e088615f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.496458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.500551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.500551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.500551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.500551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.500830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.502767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 827/1664] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:13.503880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.516402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:13.530337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154356c5-ba71-442d-a5f2-d99e17d58e12 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.538761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.539862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.539862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.540219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.541115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.543492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.636606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 828/1665] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:13.638316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 838/1666] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:13.640646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.642425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.706156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:13.788199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2971093-1d45-4732-a9d4-892592834597 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.794436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.795133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.795418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.795700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.796605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.799504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.861616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46823e33-98b4-4eb9-bf28-c369ca24ede3 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.874752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.875517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.884762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.884762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.884762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.887244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:13.925326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 839/1667] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:13.930312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:13.966373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074ddfc-8ad5-404c-9f47-1e747d425f62 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:13.979962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:13.979962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:13.980395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:13.980677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:13.981511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:13.983197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.003440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 840/1668] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:14.008156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:14.030992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b1105-0722-4f4d-b6d4-cfa8ccc2c478 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:14.039369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 829/1669] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:13 2022] POST /identity/v3/users => generated 284 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:14.040444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:14.040696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:14.041494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.041784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.042063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.042780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:14.044590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.068991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 841/1670] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:14.072975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:14.084251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1eb21-34cb-4537-81be-7aab58fc0b51 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:14.091158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:14.092086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.092458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.092827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.094522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:14.097843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.105313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bbcd4-42be-434c-97a9-667685e4f21e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:14.110908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15141dab-9840-4bc2-97f1-8ef69371c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:14.115976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141dab-9840-4bc2-97f1-8ef69371c9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.115976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141dab-9840-4bc2-97f1-8ef69371c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.115976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15141dab-9840-4bc2-97f1-8ef69371c9ac admin admin] PATH_INFO: `/v3/projects/7dd242a670e84e5dacdc696786034aa3/users/3c59f0c338194a1ba98ad33b9f2395de/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.123873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 830/1671] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:14.131818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:14.159080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af43d5d-26e4-4261-9f9b-e82051f8e7df admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:14.175452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141dab-9840-4bc2-97f1-8ef69371c9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd242a670e84e5dacdc696786034aa3, user_id=3c59f0c338194a1ba98ad33b9f2395de, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:14.178368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15141dab-9840-4bc2-97f1-8ef69371c9ac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.197773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:14.198425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.205290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.205722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.206750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:14.211192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.264471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 831/1672] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:14.268348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:14.290782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a51d5b-8fb9-4cd7-b595-99f7284b20db admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:14.299805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 842/1673] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:14 2022] PUT /identity/v3/projects/7dd242a670e84e5dacdc696786034aa3/users/3c59f0c338194a1ba98ad33b9f2395de/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:14.311063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849941a0-0226-4b4b-ade5-5d840bf5a35d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.311359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849941a0-0226-4b4b-ade5-5d840bf5a35d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.311646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849941a0-0226-4b4b-ade5-5d840bf5a35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.377445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-849941a0-0226-4b4b-ade5-5d840bf5a35d None None] MFA Rules not processed for user `3c59f0c338194a1ba98ad33b9f2395de`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:14.378704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-849941a0-0226-4b4b-ade5-5d840bf5a35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:14.421101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60539cc-025c-4c22-a97b-358a41e5fb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:14.421962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60539cc-025c-4c22-a97b-358a41e5fb36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.422988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60539cc-025c-4c22-a97b-358a41e5fb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.423370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60539cc-025c-4c22-a97b-358a41e5fb36 admin admin] PATH_INFO: `/v3/projects/8ce9591c203a44728480ce7bb3c0101f/users/f393c95f3eca4789b3d514b034476074/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.445065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60539cc-025c-4c22-a97b-358a41e5fb36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce9591c203a44728480ce7bb3c0101f, user_id=f393c95f3eca4789b3d514b034476074, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:14.448957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60539cc-025c-4c22-a97b-358a41e5fb36 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.542567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 832/1674] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:14 2022] PUT /identity/v3/projects/8ce9591c203a44728480ce7bb3c0101f/users/f393c95f3eca4789b3d514b034476074/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:14.550927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d141276-5300-45d7-b81d-f19ca171cc69 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.552284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d141276-5300-45d7-b81d-f19ca171cc69 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.553571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d141276-5300-45d7-b81d-f19ca171cc69 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.558570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 833/1675] 10.222.0.66 () {54 vars in 912 bytes} [Thu Aug 4 03:39:14 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:14.560602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.561089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.562124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.580178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 843/1676] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:14.581863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b309f2e-5337-45c8-bf84-6b03486282ee None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.584025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b309f2e-5337-45c8-bf84-6b03486282ee None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.584025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b309f2e-5337-45c8-bf84-6b03486282ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.627419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] MFA Rules not processed for user `f393c95f3eca4789b3d514b034476074`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:14.629121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:14.667396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-5b309f2e-5337-45c8-bf84-6b03486282ee None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:14.668661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5b309f2e-5337-45c8-bf84-6b03486282ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:14.743297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 834/1677] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:14.744020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:14.763224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 844/1678] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:39:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:14.771709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.772077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.772745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.782097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:14.825490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] MFA Rules not processed for user `3ec8d131a9764028a096314a4ecd58f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:14.827050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:14.887702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfb7a2-4c59-4cb6-934b-91c40b7bf219 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:14.894181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:14.895165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:14.896247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:14.896838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:14.897856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:14.900260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:14.979531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 845/1679] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:14.986985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.010130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d17b64-e754-4ee5-ab76-38e323e19057 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.015261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:15.023352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef24c6-5e15-451b-aad4-8f31c374b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.027726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 835/1680] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:14 2022] POST /identity/v3/projects => generated 385 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:15.029239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.030628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef24c6-5e15-451b-aad4-8f31c374b9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.030939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef24c6-5e15-451b-aad4-8f31c374b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.031209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef24c6-5e15-451b-aad4-8f31c374b9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.031915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef24c6-5e15-451b-aad4-8f31c374b9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.033932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef24c6-5e15-451b-aad4-8f31c374b9b6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.041663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:15.152637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 846/1681] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:14 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:15.159362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.165438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.168091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.193902 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48ece4-ce15-4cc0-9a48-afff5f90f654 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.251038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] MFA Rules not processed for user `3ec8d131a9764028a096314a4ecd58f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:15.254835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:15.312543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:15.324679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.325678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.326141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.326459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.338466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.339949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.354390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 836/1682] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:15.356312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.369388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:15.397480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 847/1683] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:15.399175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.500986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447999b9-a0f7-46ac-8921-cf7a8944d23e service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.507753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.508556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.509172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.509325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.510506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.512459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.536969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a33e3-eba1-4565-9fd0-69ea5f3539b5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.546154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ffb2e-a887-4dd5-a5a7-a4abebf474a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.546851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffb2e-a887-4dd5-a5a7-a4abebf474a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.547137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffb2e-a887-4dd5-a5a7-a4abebf474a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.547404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ffb2e-a887-4dd5-a5a7-a4abebf474a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.548889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ffb2e-a887-4dd5-a5a7-a4abebf474a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.553414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ffb2e-a887-4dd5-a5a7-a4abebf474a4 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.634124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 837/1684] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:15 2022] POST /identity/v3/users => generated 299 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:15.635336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.638531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:15.696277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 848/1685] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:15 2022] POST /identity/v3/users => generated 290 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:15.698342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.698945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.699792 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.767174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] MFA Rules not processed for user `6dba436d0a67433f8c8ae7cd40d3f795`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:15.767355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5c0cd-a49e-43a8-bbf7-63e9de4af435 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.769806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:15.774997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.775671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.775957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.776427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.803449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.807909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.833980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 838/1686] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:15.836015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.859036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50dddc5-a1a9-45e8-a089-78257e0dee94 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.865422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.866408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.866894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.867377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.868294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.871099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.884761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 839/1687] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:15.888713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.888860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:15.901580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ceb4f3-1777-40b5-935c-39789376db36 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.921771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.922577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.930318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.930318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.930318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:15.930318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:15.941980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 840/1688] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:15.943167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.944004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 849/1689] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:15.946285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:15.954140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b587e3a-2d37-4e8a-859f-1923c8225778 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.960732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.963180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.963595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.964225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] PATH_INFO: `/v3/projects/0a1936f70e524290a3ff5760743e1c7a/users/ef1e37adc1a94a8a97dd88b4b17a7470/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:15.967985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf35bf1-93d0-40c2-acec-bcd6e2b86359 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:15.987983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:15.987983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:15.987983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:15.987983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] PATH_INFO: `/v3/projects/259af367ae2f4919aeafb20d0a102d7f/users/e67afb0a10df46bfbb5f4ea664fd64eb/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.011817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1936f70e524290a3ff5760743e1c7a, user_id=ef1e37adc1a94a8a97dd88b4b17a7470, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.028376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.055319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=259af367ae2f4919aeafb20d0a102d7f, user_id=e67afb0a10df46bfbb5f4ea664fd64eb, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.062357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.171750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 841/1690] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:15 2022] PUT /identity/v3/projects/0a1936f70e524290a3ff5760743e1c7a/users/ef1e37adc1a94a8a97dd88b4b17a7470/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:16.177835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:16.193728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 850/1691] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:15 2022] PUT /identity/v3/projects/259af367ae2f4919aeafb20d0a102d7f/users/e67afb0a10df46bfbb5f4ea664fd64eb/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:16.201633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:16.281593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:16.366775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d8ba5-18ff-405c-a547-8455a0c228dc admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:16.376352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef8fad-0ca1-43c8-9a30-f9cbbe4df60d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:16.377711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:16.378379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.378638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.378915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.383037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.384776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.416413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 851/1692] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:16.431774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:16.467269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d119ff-78b1-4ea6-ade6-e2847bcba72d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:16.484333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a50c-07ae-4123-8a88-877d7f2580f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:16.485083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a50c-07ae-4123-8a88-877d7f2580f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.485410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a50c-07ae-4123-8a88-877d7f2580f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.485693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a50c-07ae-4123-8a88-877d7f2580f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.486407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a50c-07ae-4123-8a88-877d7f2580f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.491768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a50c-07ae-4123-8a88-877d7f2580f5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.513637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 852/1693] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:16.516817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938efd0-af8a-4943-9856-ee028a1d3a79 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.516817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938efd0-af8a-4943-9856-ee028a1d3a79 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.517232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938efd0-af8a-4943-9856-ee028a1d3a79 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.520048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 853/1694] 10.222.0.66 () {54 vars in 961 bytes} [Thu Aug 4 03:39:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:16.522599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938efd0-af8a-4943-9856-ee028a1d3a79 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:16.535884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ca2c1-8dca-4624-953e-477c54d916a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:16.536716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ca2c1-8dca-4624-953e-477c54d916a7 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.537125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ca2c1-8dca-4624-953e-477c54d916a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.537537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ca2c1-8dca-4624-953e-477c54d916a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.541730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938efd0-af8a-4943-9856-ee028a1d3a79 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:16.554119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d2e3e-7789-4c9c-9b1b-35a7aac1bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:16.555025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d2e3e-7789-4c9c-9b1b-35a7aac1bcaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.555619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d2e3e-7789-4c9c-9b1b-35a7aac1bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.555882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d2e3e-7789-4c9c-9b1b-35a7aac1bcaa admin admin] PATH_INFO: `/v3/projects/0a1936f70e524290a3ff5760743e1c7a/users/ef1e37adc1a94a8a97dd88b4b17a7470/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.559029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d2e3e-7789-4c9c-9b1b-35a7aac1bcaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1936f70e524290a3ff5760743e1c7a, user_id=ef1e37adc1a94a8a97dd88b4b17a7470, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.562673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d2e3e-7789-4c9c-9b1b-35a7aac1bcaa admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.589734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 854/1695] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:16 2022] PUT /identity/v3/projects/0a1936f70e524290a3ff5760743e1c7a/users/ef1e37adc1a94a8a97dd88b4b17a7470/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:16.589734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c287-663d-490e-ab96-c0becddca560 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.589734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c287-663d-490e-ab96-c0becddca560 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.589734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019c287-663d-490e-ab96-c0becddca560 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.589734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 855/1696] 10.222.0.66 () {54 vars in 912 bytes} [Thu Aug 4 03:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:16.590362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c287-663d-490e-ab96-c0becddca560 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:16.637761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ca2c1-8dca-4624-953e-477c54d916a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.641131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ca2c1-8dca-4624-953e-477c54d916a7 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.668647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 842/1697] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:16.672109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4cb80-1f65-421f-ad9b-f98cacd4dcc4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.672696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4cb80-1f65-421f-ad9b-f98cacd4dcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.672799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4cb80-1f65-421f-ad9b-f98cacd4dcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.705977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9019c287-663d-490e-ab96-c0becddca560 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:16.712226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5b931-0254-4127-89d2-7bcef1f95a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:16.714028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5b931-0254-4127-89d2-7bcef1f95a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.714422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5b931-0254-4127-89d2-7bcef1f95a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.714850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5b931-0254-4127-89d2-7bcef1f95a94 admin admin] PATH_INFO: `/v3/projects/259af367ae2f4919aeafb20d0a102d7f/users/e67afb0a10df46bfbb5f4ea664fd64eb/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.716911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-daf4cb80-1f65-421f-ad9b-f98cacd4dcc4 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:16.717751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5b931-0254-4127-89d2-7bcef1f95a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259af367ae2f4919aeafb20d0a102d7f, user_id=e67afb0a10df46bfbb5f4ea664fd64eb, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:16.719309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-daf4cb80-1f65-421f-ad9b-f98cacd4dcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:16.721536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5b931-0254-4127-89d2-7bcef1f95a94 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:16.745469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 856/1698] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:16 2022] PUT /identity/v3/projects/259af367ae2f4919aeafb20d0a102d7f/users/e67afb0a10df46bfbb5f4ea664fd64eb/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:16.745469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ac449-6efd-48ac-bc91-eb1d6527d417 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.745469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ac449-6efd-48ac-bc91-eb1d6527d417 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.745469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ac449-6efd-48ac-bc91-eb1d6527d417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.777757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9e3ac449-6efd-48ac-bc91-eb1d6527d417 None None] MFA Rules not processed for user `ef1e37adc1a94a8a97dd88b4b17a7470`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:16.779191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9e3ac449-6efd-48ac-bc91-eb1d6527d417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:16.803602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:16.826260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 843/1699] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Aug 4 03:39:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:16.827649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.827942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.828216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.870604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:16.888543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:16.891893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:16.933765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 857/1700] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:16.935215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:16.936956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:16.937354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:16.977747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] MFA Rules not processed for user `e67afb0a10df46bfbb5f4ea664fd64eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:16.979074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:17.017399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 844/1701] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:39:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:17.018961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:17.085954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 858/1702] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:17.085954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:17.085954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:17.136224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff285d-f07b-4be8-a729-1fc69b991a61 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:17.150202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:17.151039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:17.164588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:17.164588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:17.164588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:17.215054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:17.218378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:17.261478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a202e-cbe4-4259-b072-d570a75f28c9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:17.272747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:17.273523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:17.273883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:17.274233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:17.279040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 845/1703] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:17.281953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:17.296108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:17.298490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:17.313986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c217980-e94e-4c3d-94df-23f91b11578b service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:17.320982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:17.321734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:17.323873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:17.324484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:17.336113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 859/1704] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:17.336947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:17.337354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:17.339212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:17.339324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:17.351705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 846/1705] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:17.364291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396e7a7-842a-42d9-9340-85369ccc5508 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:17.376786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bcc0f-40b5-45de-91c2-8a45ff65e12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:17.378034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bcc0f-40b5-45de-91c2-8a45ff65e12e service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:17.378034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bcc0f-40b5-45de-91c2-8a45ff65e12e service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:17.378034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bcc0f-40b5-45de-91c2-8a45ff65e12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:17.405237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bcc0f-40b5-45de-91c2-8a45ff65e12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:17.410500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bcc0f-40b5-45de-91c2-8a45ff65e12e service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:17.447238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 860/1706] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:17.642962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:17.643878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:17.798469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7690fe8-f5cd-4e17-b9d9-3dea432e0217 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:17.825189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:17.825877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:17.826155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:17.826421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] PATH_INFO: `/v3/users/6daf35a4dd4249ecb83225047b942998` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:17.830940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6daf35a4dd4249ecb83225047b942998)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:17.833874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:18.085237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:18.124718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6daf35a4dd4249ecb83225047b942998'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:18.157133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6daf35a4dd4249ecb83225047b942998'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:18.157133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6daf35a4dd4249ecb83225047b942998'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:18.211181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6daf35a4dd4249ecb83225047b942998'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:18.293552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6daf35a4dd4249ecb83225047b942998'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:18.306398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 847/1707] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:17 2022] DELETE /identity/v3/users/6daf35a4dd4249ecb83225047b942998 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:18.312849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a171f-d399-42fe-8fdd-713a23213e16 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:18.313344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a171f-d399-42fe-8fdd-713a23213e16 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:18.313738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a171f-d399-42fe-8fdd-713a23213e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:18.371167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6f8a171f-d399-42fe-8fdd-713a23213e16 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:18.375980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6f8a171f-d399-42fe-8fdd-713a23213e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:18.439611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:18.531568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 861/1708] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:18.542926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:18.550970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:18.641608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9a6f-8ff5-4c3e-9074-ee5b9b304f71 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:18.641907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9a6f-8ff5-4c3e-9074-ee5b9b304f71 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:18.642225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd9a6f-8ff5-4c3e-9074-ee5b9b304f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:18.694767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-60dd9a6f-8ff5-4c3e-9074-ee5b9b304f71 None None] MFA Rules not processed for user `3ec8d131a9764028a096314a4ecd58f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:18.703014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-60dd9a6f-8ff5-4c3e-9074-ee5b9b304f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:18.745862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfc30d-adde-4e5d-b5e3-1e97f40ca1ed admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:18.758592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:18.759162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:18.760170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:18.760906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:18.807639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:18.810939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:18.847136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 848/1709] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:18.896238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 862/1710] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:19.792008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:19.793624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:19.825598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431bd84-afb2-49f1-8e65-56f41db0249e service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:19.857093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:19.860970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:19.861291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:19.864881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] PATH_INFO: `/v3/projects/d3ace7ff4e72453bab404b46bb8b2cef` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:19.875578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ace7ff4e72453bab404b46bb8b2cef)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:19.876206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:20.090512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ace7ff4e72453bab404b46bb8b2cef', 'request_id': 'req-0bd738ea-9317-4339-9bc5-a30613788c50'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:20.102094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ace7ff4e72453bab404b46bb8b2cef', 'request_id': 'req-0bd738ea-9317-4339-9bc5-a30613788c50'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:20.226994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ace7ff4e72453bab404b46bb8b2cef', 'request_id': 'req-0bd738ea-9317-4339-9bc5-a30613788c50'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:20.263395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] The token cache is being invalidate because project d3ace7ff4e72453bab404b46bb8b2cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:20.263903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:20.266252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0bd738ea-9317-4339-9bc5-a30613788c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:20.271975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 849/1711] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:19 2022] DELETE /identity/v3/projects/d3ace7ff4e72453bab404b46bb8b2cef => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:20.276888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:20.277482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:20.277482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:20.321310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:20.339224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:20.545579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 863/1712] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:20.554042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748e47d-6057-461f-96f1-492054614840 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:20.554562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748e47d-6057-461f-96f1-492054614840 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:20.554983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748e47d-6057-461f-96f1-492054614840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:20.586141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d748e47d-6057-461f-96f1-492054614840 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:20.587817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d748e47d-6057-461f-96f1-492054614840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:20.597292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 850/1713] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:20.603188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:20.622189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14720691-9df2-4668-9cc0-c864d51e4164 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:20.648022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:20.651126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:20.651126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:20.651126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:20.654601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:20.654601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:20.657241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:20.690177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 864/1714] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:20.691372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748e47d-6057-461f-96f1-492054614840 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:20.717459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748e47d-6057-461f-96f1-492054614840 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:20.725463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:20.728151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:20.728592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:20.729019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:20.729878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:20.732338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:20.822091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:20.822091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:20.839071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373afb-cb1d-46ef-bc42-5bc0ede38cdc admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:20.845082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:20.850466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:20.850466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:20.850466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] PATH_INFO: `/v3/users/3c59f0c338194a1ba98ad33b9f2395de` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:20.855010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c59f0c338194a1ba98ad33b9f2395de)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:20.860313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:20.886665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 851/1715] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:20 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:20.890074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:20.949148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:21.163314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c59f0c338194a1ba98ad33b9f2395de'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.163314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c59f0c338194a1ba98ad33b9f2395de'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.164263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c59f0c338194a1ba98ad33b9f2395de'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.177798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7657f-ea03-43d8-b758-dacdb236def7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:21.184404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:21.187886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ca10f-5ae7-4dea-bd72-907dd895f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:21.197796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ca10f-5ae7-4dea-bd72-907dd895f990 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:21.197796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ca10f-5ae7-4dea-bd72-907dd895f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:21.197796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ca10f-5ae7-4dea-bd72-907dd895f990 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:21.198137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c59f0c338194a1ba98ad33b9f2395de'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.198234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ca10f-5ae7-4dea-bd72-907dd895f990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:21.211769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ca10f-5ae7-4dea-bd72-907dd895f990 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:21.288863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c59f0c338194a1ba98ad33b9f2395de'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.310903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 865/1716] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:20 2022] DELETE /identity/v3/users/3c59f0c338194a1ba98ad33b9f2395de => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:21.312220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:21.319912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:21.337953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 852/1717] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:20 2022] POST /identity/v3/users => generated 282 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:21.342743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:21.344625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:21.345158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:21.378599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:21.381003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:21.509640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10509d16-7b30-4ca3-915c-8d7317966f9e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:21.522363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:21.523752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:21.524040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:21.524322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] PATH_INFO: `/v3/users/3719507c161845789093a0c70189448f` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:21.525565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3719507c161845789093a0c70189448f)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:21.531405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:21.540183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 853/1718] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:21.540183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:21.550135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:21.577648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24471-3f46-408e-bac0-ba85860fa4bb None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:21.601281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:21.601281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:21.601281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:21.607615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:21.611704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:21.617729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:21.653679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 854/1719] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:21.658674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:21.665636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:21.871152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3719507c161845789093a0c70189448f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.872527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3719507c161845789093a0c70189448f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.874309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfbd1-c630-4176-963a-a5d060dcfac0 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:21.875927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3719507c161845789093a0c70189448f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.884181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:21.884971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:21.885422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:21.885692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:21.888826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:21.926704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3719507c161845789093a0c70189448f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:21.940576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:21.942152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:22.016297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 855/1720] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:22.048138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c377752c-91e7-4cf3-a5b8-7eff1614403c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3719507c161845789093a0c70189448f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:22.048354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:22.058947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:22.079863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 866/1721] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:21 2022] DELETE /identity/v3/users/3719507c161845789093a0c70189448f => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:22.089177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:22.091167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:22.091167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:22.093499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5208f4-c60f-448d-9005-2d54a6ce52a1 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:22.161130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:22.172343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:22.203387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:22.333893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:22.345238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcd2c2-d917-41be-ac0b-04de59183252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:22.346840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd2c2-d917-41be-ac0b-04de59183252 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:22.347664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd2c2-d917-41be-ac0b-04de59183252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:22.349377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcd2c2-d917-41be-ac0b-04de59183252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:22.350630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcd2c2-d917-41be-ac0b-04de59183252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:22.355662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcd2c2-d917-41be-ac0b-04de59183252 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:22.372698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 867/1722] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:22.389421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 856/1723] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:22.392715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:22.423526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784a6f-2bac-4ad4-b676-a7531297926b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:22.435811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:22.437991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:22.439445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:22.439743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] PATH_INFO: `/v3/projects/b6398c32130945ed8d4f59d770b4b41c/users/5900aaca6e5f47b196e5fb56e11b1521/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:22.491227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6398c32130945ed8d4f59d770b4b41c, user_id=5900aaca6e5f47b196e5fb56e11b1521, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:22.500037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:22.614483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 868/1724] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:22 2022] PUT /identity/v3/projects/b6398c32130945ed8d4f59d770b4b41c/users/5900aaca6e5f47b196e5fb56e11b1521/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:22.622361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f793a0-f2b4-448f-9f6b-44abc94937e0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:22.622653 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f793a0-f2b4-448f-9f6b-44abc94937e0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:22.622934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f793a0-f2b4-448f-9f6b-44abc94937e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:22.632607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:22.638615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:22.734565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-35f793a0-f2b4-448f-9f6b-44abc94937e0 None None] MFA Rules not processed for user `5900aaca6e5f47b196e5fb56e11b1521`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:22.735894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-35f793a0-f2b4-448f-9f6b-44abc94937e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:22.835067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23eeac-e1f1-4fe4-8266-0493d9bac47a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:22.852349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:22.860543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:22.860543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:22.860543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:22.958404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 857/1725] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:22.960541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73a478-e7ce-4395-a84a-36386760c316 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:22.962364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73a478-e7ce-4395-a84a-36386760c316 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:22.962809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73a478-e7ce-4395-a84a-36386760c316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:23.033698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ba73a478-e7ce-4395-a84a-36386760c316 None None] MFA Rules not processed for user `6dba436d0a67433f8c8ae7cd40d3f795`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:23.035316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ba73a478-e7ce-4395-a84a-36386760c316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:23.050167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:23.052118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:23.082074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 869/1726] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:39:23.083612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:23.140181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e29f77-66f9-4426-85bc-c8467c0d2799 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:23.167331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:23.168038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:23.168323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:23.168598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:23.169645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:23.173392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:23.265853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 858/1727] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:23.360425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:23.360711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:23.360988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:23.380827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 870/1728] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:23 2022] POST /identity/v3/projects => generated 363 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:23.385971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:23.608862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:23.619732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b710-4736-4775-8865-5c93819e7719 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:23.625344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:23.626114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:23.626498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:23.626866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:23.627657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:23.632101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:23.633124 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:23.634574 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:23.722317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:23.794236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 859/1729] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:39:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:23.925072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 871/1730] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:23 2022] POST /identity/v3/users => generated 288 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:23.929743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:23.960435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:23.963615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:23.973892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a89ca0-db44-42d0-855e-fc4155e60803 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:23.979059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:23.980036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:23.980456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:23.980890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:23.981757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:23.983539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:23.984251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26539d-f348-4cb1-a3db-ba02a7b5c641 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:23.990747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:23.991489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:23.991848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:23.992199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] PATH_INFO: `/v3/users/f393c95f3eca4789b3d514b034476074` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:23.993519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f393c95f3eca4789b3d514b034476074)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:23.995783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:24.007082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 860/1731] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:24.016032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:24.033832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288edda9-4dbc-491f-a92c-383fc58b5ff8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:24.040838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:24.041906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:24.043937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:24.044481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] PATH_INFO: `/v3/projects/789f1120bc59404c87a0973a92a39fe3/users/aafc8bbb125742f08e1d533435b2f66b/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:24.066353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=789f1120bc59404c87a0973a92a39fe3, user_id=aafc8bbb125742f08e1d533435b2f66b, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:24.070743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:24.219646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 861/1732] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:24 2022] PUT /identity/v3/projects/789f1120bc59404c87a0973a92a39fe3/users/aafc8bbb125742f08e1d533435b2f66b/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:24.221028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:24.226491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:24.345436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f393c95f3eca4789b3d514b034476074'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.349591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f393c95f3eca4789b3d514b034476074'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.350318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f393c95f3eca4789b3d514b034476074'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.352058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2234462-b8ad-4827-aa63-6cc3a4dbd462 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:24.367662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:24.372226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:24.372498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:24.372796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] PATH_INFO: `/v3/projects/7dd242a670e84e5dacdc696786034aa3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:24.374860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd242a670e84e5dacdc696786034aa3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:24.376709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f393c95f3eca4789b3d514b034476074'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.378972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:24.494558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f393c95f3eca4789b3d514b034476074'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.524915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 872/1733] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:23 2022] DELETE /identity/v3/users/f393c95f3eca4789b3d514b034476074 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:24.526302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:24.528104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:24.538560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd242a670e84e5dacdc696786034aa3', 'request_id': 'req-403266c2-cd5e-4ce0-8e9d-199426dc1fda'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.556244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd242a670e84e5dacdc696786034aa3', 'request_id': 'req-403266c2-cd5e-4ce0-8e9d-199426dc1fda'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.633974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d9e53-01c1-4e9d-af75-d525cbb353f6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:24.640403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:24.641539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:24.641896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:24.642683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:24.643010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:24.647790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd242a670e84e5dacdc696786034aa3', 'request_id': 'req-403266c2-cd5e-4ce0-8e9d-199426dc1fda'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.655378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:24.674939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] The token cache is being invalidate because project 7dd242a670e84e5dacdc696786034aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:24.677984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 873/1734] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:24.680429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.682258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-403266c2-cd5e-4ce0-8e9d-199426dc1fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:24.686540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:24.691065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 862/1735] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:24 2022] DELETE /identity/v3/projects/7dd242a670e84e5dacdc696786034aa3 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:24.692656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:24.696952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:24.701867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:24.702537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:24.751949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:24.756215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:24.767868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 863/1736] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:24.769107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:24.772990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:24.795230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9919f-ee7a-4361-b15e-379a3655a604 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:24.804528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:24.804956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:24.806627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:24.806907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:24.807283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] PATH_INFO: `/v3/projects/789f1120bc59404c87a0973a92a39fe3/users/aafc8bbb125742f08e1d533435b2f66b/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:24.809740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=789f1120bc59404c87a0973a92a39fe3, user_id=aafc8bbb125742f08e1d533435b2f66b, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:24.812846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:24.841527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 864/1737] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:24 2022] PUT /identity/v3/projects/789f1120bc59404c87a0973a92a39fe3/users/aafc8bbb125742f08e1d533435b2f66b/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:24.844742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:24.846270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:24.865069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:24.892908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7c577-8c1a-4923-b05b-0d9cffba7a7a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:24.972123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de652267-6e06-40d1-a695-c20e43f2fb60 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:24.984149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:24.984797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:24.985049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:24.986167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] PATH_INFO: `/v3/users/ef1e37adc1a94a8a97dd88b4b17a7470` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:24.989786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1e37adc1a94a8a97dd88b4b17a7470)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:24.998338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:24.999004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:25.002252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:25.002252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:25.002862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:25.008850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:25.147024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:25.148804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:25.159149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 874/1738] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:39:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:39:25.159861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:25.171181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:25.338375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1e37adc1a94a8a97dd88b4b17a7470'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:25.377421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1e37adc1a94a8a97dd88b4b17a7470'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:25.380085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1e37adc1a94a8a97dd88b4b17a7470'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:25.417659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17695b24-564e-43d6-9dff-1f2a2255868d service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:25.432278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:25.432851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:25.433136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:25.433436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:25.441126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1e37adc1a94a8a97dd88b4b17a7470'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:25.447397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:25.471034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:25.475696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e28db-1b2a-4a95-82ff-00931784046a service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:25.505302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1e37adc1a94a8a97dd88b4b17a7470'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:25.507369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 875/1739] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:25.508421 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed203752-337d-477d-b491-6a21ba33156f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:25.508709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed203752-337d-477d-b491-6a21ba33156f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:25.508988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed203752-337d-477d-b491-6a21ba33156f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:25.515962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 865/1740] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:24 2022] DELETE /identity/v3/users/ef1e37adc1a94a8a97dd88b4b17a7470 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:25.515962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:25.515962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:25.619676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ed203752-337d-477d-b491-6a21ba33156f None None] MFA Rules not processed for user `aafc8bbb125742f08e1d533435b2f66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:25.620924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ed203752-337d-477d-b491-6a21ba33156f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:25.731009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:25.838603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6347ae-c703-46b6-b8ed-e4949fdcb050 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:25.854279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:25.855785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:25.857687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:25.858130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] PATH_INFO: `/v3/projects/08ca5027abb74e11ac8c337217852767` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:25.860114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ca5027abb74e11ac8c337217852767)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:25.880479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:25.954182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 876/1741] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:25.957492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:25.957492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:25.957492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:26.000245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:26.001825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:26.169400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 877/1742] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Aug 4 03:39:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:26.170283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:26.175169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:26.292487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:26.314428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc069020-c3ea-458a-8a94-cb71efec4d74 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:26.386450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ca5027abb74e11ac8c337217852767', 'request_id': 'req-6330446a-fa7f-4411-8582-861002226705'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:26.399817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ca5027abb74e11ac8c337217852767', 'request_id': 'req-6330446a-fa7f-4411-8582-861002226705'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:26.464157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:26.470956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:26.470956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:26.470956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:26.533937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:26.630768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:26.632518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:26.643486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ca5027abb74e11ac8c337217852767', 'request_id': 'req-6330446a-fa7f-4411-8582-861002226705'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:26.664481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] The token cache is being invalidate because project 08ca5027abb74e11ac8c337217852767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:26.666507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:26.666507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:26.668802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 866/1743] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:25 2022] DELETE /identity/v3/projects/08ca5027abb74e11ac8c337217852767 => generated 0 bytes in 1155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:26.669774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:26.674206 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:26.690673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 878/1744] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:26.693473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:26.699176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:26.804210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330446a-fa7f-4411-8582-861002226705 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:26.812742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:26.815640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:26.815640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:26.815640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:26.822035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:26.828314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75817d-fe2c-4e2c-b83a-c5b1372e344d service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:26.833524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d9b26-ea3e-4bb2-8751-0281a9178870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:26.834339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d9b26-ea3e-4bb2-8751-0281a9178870 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:26.834694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d9b26-ea3e-4bb2-8751-0281a9178870 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:26.835041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d9b26-ea3e-4bb2-8751-0281a9178870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:26.864795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d9b26-ea3e-4bb2-8751-0281a9178870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:26.866861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d9b26-ea3e-4bb2-8751-0281a9178870 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:26.871743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:26.878274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:26.878694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 879/1745] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:26.880403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9bc72-0a7b-403a-81b2-d6a641fb696a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:26.880835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9bc72-0a7b-403a-81b2-d6a641fb696a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:26.881280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9bc72-0a7b-403a-81b2-d6a641fb696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:26.903067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3ba9bc72-0a7b-403a-81b2-d6a641fb696a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:26.905065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3ba9bc72-0a7b-403a-81b2-d6a641fb696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:26.920355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 867/1746] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:27.054626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:27.056151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:27.064675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 880/1747] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:27.066146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.066451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.066715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.091186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87429be5-8550-465b-8a98-58dcaf8991f8 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:27.103030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:27.104153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.104598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.105019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.107197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:27.128049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:27.130572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:27.153909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 881/1748] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:27.158050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:27.183513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4e336-ecc2-4e42-b6c8-0fe6bb059da8 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:27.195160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:27.195770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.196058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.196333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.197749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:27.198067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:27.212201 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:27.237801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:27.240641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:27.256933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 868/1749] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:27.257723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 882/1750] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:39:27.272813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:27.272813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b11cc3-f5bd-4180-a384-d9b2ea8c51dd service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:27.283384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:27.285430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.285837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.286404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.289441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:27.292543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:27.457182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 869/1751] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:27 2022] POST /identity/v3/projects => generated 373 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:27.461543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:27.483756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc977-8c6a-4fb1-b732-a9baa774eae1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:27.639475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:27.639475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:27.666960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:27.691260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:27.696278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.696593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.696878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.697752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:27.703796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:27.753847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:27.837601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203151b-c8e6-42ab-990d-e4be9d71b241 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:27.848064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:27.857380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.858530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.862826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] PATH_INFO: `/v3/users/7485417b59cd4b2184f52d99435728ee` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.868459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 883/1752] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:27 2022] POST /identity/v3/users => generated 287 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:27.871065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:27.872491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:27.881529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7485417b59cd4b2184f52d99435728ee)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:27.886351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:27.923637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3375b83-e561-46e5-9a55-1d6b3e6e14ee admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:27.937617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:27.938275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:27.938566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:27.938852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] PATH_INFO: `/v3/projects/0a1936f70e524290a3ff5760743e1c7a` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:27.944261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1936f70e524290a3ff5760743e1c7a)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:27.951072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:28.342572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7485417b59cd4b2184f52d99435728ee'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.351606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7485417b59cd4b2184f52d99435728ee'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.352354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7485417b59cd4b2184f52d99435728ee'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.367523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1936f70e524290a3ff5760743e1c7a', 'request_id': 'req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.376496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7485417b59cd4b2184f52d99435728ee'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.386640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1936f70e524290a3ff5760743e1c7a', 'request_id': 'req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.556016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7485417b59cd4b2184f52d99435728ee'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.568547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 870/1753] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:27 2022] DELETE /identity/v3/users/7485417b59cd4b2184f52d99435728ee => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:28.568547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:28.570464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:28.729401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1936f70e524290a3ff5760743e1c7a', 'request_id': 'req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.751247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] The token cache is being invalidate because project 0a1936f70e524290a3ff5760743e1c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:28.755701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.755701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:28.755701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 884/1754] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:27 2022] DELETE /identity/v3/projects/0a1936f70e524290a3ff5760743e1c7a => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:28.756422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:28.758354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:28.759833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26a4e3-00b7-4748-8b24-b07f585b9818 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:28.774045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013c7e6-e417-4b77-8ff1-8cde2cf19af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:28.774701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c7e6-e417-4b77-8ff1-8cde2cf19af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:28.774987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c7e6-e417-4b77-8ff1-8cde2cf19af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:28.775262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c7e6-e417-4b77-8ff1-8cde2cf19af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:28.775980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013c7e6-e417-4b77-8ff1-8cde2cf19af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:28.777755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013c7e6-e417-4b77-8ff1-8cde2cf19af3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:28.799126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 871/1755] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:28.800853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2e32-798c-4fc2-b45b-2beec187c026 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:28.801908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2e32-798c-4fc2-b45b-2beec187c026 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:28.802816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2e32-798c-4fc2-b45b-2beec187c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:28.839297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a63c2e32-798c-4fc2-b45b-2beec187c026 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:28.840673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a63c2e32-798c-4fc2-b45b-2beec187c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:28.859632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 872/1756] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:28.859632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:28.860105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:28.860526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:28.895780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:28.897660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:28.915017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 873/1757] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:28.918730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:28.925553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:28.925795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ce1b6-ca62-4f8e-8a61-896dcc2e5ff6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:28.938144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:28.938823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:28.939104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:28.939376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] PATH_INFO: `/v3/projects/8ce9591c203a44728480ce7bb3c0101f` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:28.946246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce9591c203a44728480ce7bb3c0101f)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:28.948050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba00c58-8523-4530-a0a5-ed3cff3d450e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:28.948260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:28.956935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:28.959284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:28.959825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:28.960306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:28.962438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:28.964782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:28.986364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 874/1758] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:28.988469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:28.992406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:29.041999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:29.093279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b4d2-4e42-409d-91aa-2105e7177829 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:29.109004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc37b2-ee49-4816-b192-599a9dc15ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:29.110348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc37b2-ee49-4816-b192-599a9dc15ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.110831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc37b2-ee49-4816-b192-599a9dc15ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.111294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc37b2-ee49-4816-b192-599a9dc15ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.149091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc37b2-ee49-4816-b192-599a9dc15ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:29.155877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc37b2-ee49-4816-b192-599a9dc15ab4 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:29.166554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 875/1759] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:29.166768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce9591c203a44728480ce7bb3c0101f', 'request_id': 'req-087a742a-276a-4da7-b308-05d722ae6235'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.168683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf9c7c-ddf4-46c2-927f-52ff48b31a67 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.169156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf9c7c-ddf4-46c2-927f-52ff48b31a67 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.169627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf9c7c-ddf4-46c2-927f-52ff48b31a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.193921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce9591c203a44728480ce7bb3c0101f', 'request_id': 'req-087a742a-276a-4da7-b308-05d722ae6235'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.196042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-19cf9c7c-ddf4-46c2-927f-52ff48b31a67 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:29.198123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-19cf9c7c-ddf4-46c2-927f-52ff48b31a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:29.272391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce9591c203a44728480ce7bb3c0101f', 'request_id': 'req-087a742a-276a-4da7-b308-05d722ae6235'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.274052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=83,keystone:INSERT=5,keystone:DELETE=12 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:29.291802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] The token cache is being invalidate because project 8ce9591c203a44728480ce7bb3c0101f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:29.292196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.293103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.297761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 885/1760] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:28 2022] DELETE /identity/v3/projects/8ce9591c203a44728480ce7bb3c0101f => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:29.297761 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:29.303370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:29.337896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 876/1761] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:29.339526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6863a-d653-4d10-80ed-6676801035c9 None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.340010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6863a-d653-4d10-80ed-6676801035c9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.340472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6863a-d653-4d10-80ed-6676801035c9 None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.342001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 877/1762] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:29.351736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a742a-276a-4da7-b308-05d722ae6235 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:29.353628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6863a-d653-4d10-80ed-6676801035c9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:29.364902 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2ed6863a-d653-4d10-80ed-6676801035c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:29.373280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:29.374235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.374517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.377822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] PATH_INFO: `/v3/projects/e3013ec19294499c8b6411b149c17817/users/14642b529b214adb97178f23e5e3503d/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.394172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6863a-d653-4d10-80ed-6676801035c9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:29.405306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3013ec19294499c8b6411b149c17817, user_id=14642b529b214adb97178f23e5e3503d, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:29.408598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:29.409324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.412758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.412758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] PATH_INFO: `/v3/users/e67afb0a10df46bfbb5f4ea664fd64eb` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.412758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67afb0a10df46bfbb5f4ea664fd64eb)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:29.413900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:29.419877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:29.564072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 886/1763] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:29 2022] PUT /identity/v3/projects/e3013ec19294499c8b6411b149c17817/users/14642b529b214adb97178f23e5e3503d/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:29.564072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:29.695000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67afb0a10df46bfbb5f4ea664fd64eb'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.726635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67afb0a10df46bfbb5f4ea664fd64eb'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.727399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67afb0a10df46bfbb5f4ea664fd64eb'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.762604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67afb0a10df46bfbb5f4ea664fd64eb'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.834767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48eb4-09b0-4b61-b7af-49d73bbb487a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:29.857535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:29.858477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.858477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.861569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.862616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:29.862927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:29.868602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb3b99-0a66-4800-8080-6c2f1839165a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:29.887049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6c8bc69d-d6f0-4a6f-adf7-f65a3bac3dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67afb0a10df46bfbb5f4ea664fd64eb'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:29.902002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 878/1764] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:29 2022] DELETE /identity/v3/users/e67afb0a10df46bfbb5f4ea664fd64eb => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:29.906605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.907929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.908371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.915686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 887/1765] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:29.933515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad8d8b-105b-4d1f-a546-a3e1cf850324 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.933515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad8d8b-105b-4d1f-a546-a3e1cf850324 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.933515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad8d8b-105b-4d1f-a546-a3e1cf850324 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.933515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 888/1766] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:39:29 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:29.933515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3302f41-1054-495f-a9e7-3521c176d3eb None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:29.933515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3302f41-1054-495f-a9e7-3521c176d3eb None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:29.934502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3302f41-1054-495f-a9e7-3521c176d3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:29.969266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:29.981763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:30.071559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a3302f41-1054-495f-a9e7-3521c176d3eb None None] MFA Rules not processed for user `14642b529b214adb97178f23e5e3503d`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:30.073795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a3302f41-1054-495f-a9e7-3521c176d3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:30.105123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 879/1767] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:39:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:30.106583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:30.109478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:30.238430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bdcba-58c6-44ca-9192-2b22f8b790c1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:30.260093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:30.260755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:30.261039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:30.261345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:30.262040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:30.264418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:30.375713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 880/1768] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:30 2022] POST /identity/v3/projects => generated 371 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:30.377352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 889/1769] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:30.378944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:30.379918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:30.380347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:30.380831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:30.386016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:30.434341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:30.436487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:30.608123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcefc8c-cb2c-468f-a589-754980c16cab admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:30.618427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 890/1770] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:39:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:30.621928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:30.628168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:30.631796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:30.632803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:30.649426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:30.714093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:30.714093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:30.717030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 881/1771] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:30.718745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:30.721246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:30.821900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2c538-a08c-4c4c-ab5f-e4dae1d3ef3f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:30.836083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:30.836909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:30.837193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:30.839010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:30.841329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:30.854134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:30.870294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23214361-83d3-4be4-a63d-0c73125cf1a6 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:30.878476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:30.879062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:30.879700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:30.880030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:30.880802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:30.884447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:30.988072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 891/1772] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:30 2022] POST /identity/v3/users => generated 286 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:30.989566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:30.994476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:31.050241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 882/1773] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:30 2022] POST /identity/v3/projects => generated 373 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:31.051834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.060365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01757-903f-40a8-9c3b-b2e206274da1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.063358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:31.065713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:31.217069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65a43f-9b47-4d12-a19a-850425c888a6 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.218432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.219196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.219489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.221832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] PATH_INFO: `/v3/projects/1f44dc3db02e4cadbdb11bf0c87d1a4d` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.226270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f44dc3db02e4cadbdb11bf0c87d1a4d)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.233814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.239305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.239305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.239305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.239305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.260683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.264017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.283121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 883/1774] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:31.284003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.299521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb03a2-783d-4cdb-a118-282f34381dc3 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.313544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.320139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.322585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:31.323288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.323582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.325834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.333851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.353532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 884/1775] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:31.354713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.378532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbe8c1-7868-43bc-a222-19b35da92baa admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.385048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.385554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.386004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.386288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.387026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.389133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.432351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f44dc3db02e4cadbdb11bf0c87d1a4d', 'request_id': 'req-03090606-d88d-43b1-b926-10abff8759e3'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:31.442462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f44dc3db02e4cadbdb11bf0c87d1a4d', 'request_id': 'req-03090606-d88d-43b1-b926-10abff8759e3'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:31.478815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 885/1776] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:31 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:31.479640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.508342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f66b1-1e01-4121-91f7-939ba9de01ef admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.558412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f44dc3db02e4cadbdb11bf0c87d1a4d', 'request_id': 'req-03090606-d88d-43b1-b926-10abff8759e3'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:31.581416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] The token cache is being invalidate because project 1f44dc3db02e4cadbdb11bf0c87d1a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:31.583081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:31.585450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:31.599371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 892/1777] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:30 2022] DELETE /identity/v3/projects/1f44dc3db02e4cadbdb11bf0c87d1a4d => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:31.599371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.599371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:31.665854 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b51e10-7941-46b5-ac50-629a18af921e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.666437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b51e10-7941-46b5-ac50-629a18af921e admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.666753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b51e10-7941-46b5-ac50-629a18af921e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.667044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b51e10-7941-46b5-ac50-629a18af921e admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.667749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b51e10-7941-46b5-ac50-629a18af921e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.669647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b51e10-7941-46b5-ac50-629a18af921e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.705184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 886/1778] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:31.706502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.706793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.707070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.714370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03090606-d88d-43b1-b926-10abff8759e3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.730200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cced8e-bdf7-49ee-9555-593d7b416bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.733061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cced8e-bdf7-49ee-9555-593d7b416bec admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.735062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cced8e-bdf7-49ee-9555-593d7b416bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.735062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cced8e-bdf7-49ee-9555-593d7b416bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.735693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cced8e-bdf7-49ee-9555-593d7b416bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.738523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:31.740822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:31.741449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cced8e-bdf7-49ee-9555-593d7b416bec admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.763290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 887/1779] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:31.764114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.768196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 893/1780] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:31.778136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.778416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.778833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.791436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:31.828535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66af864-2dbd-496b-8a9a-e028ee54b522 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.837292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:31.841141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:31.850639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.851364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.851618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.851994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] PATH_INFO: `/v3/projects/d1f75c2759794117a66f839dc2479912/users/7cde8c3305c84856835c9a7f11227157/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.871088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 894/1781] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:31.872155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:31.897181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403aced-e32b-4cc1-9134-90785fdbd906 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:31.901785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f75c2759794117a66f839dc2479912, user_id=7cde8c3305c84856835c9a7f11227157, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.905662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:31.907619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:31.911540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:31.912405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:31.912938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] PATH_INFO: `/v3/projects/f516683cd3344c45adbdf92d448060c4/users/f80cbb2051af4224ad2a48c28ffde066/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:31.958387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f516683cd3344c45adbdf92d448060c4, user_id=f80cbb2051af4224ad2a48c28ffde066, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:31.958387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:32.035441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 888/1782] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:31 2022] PUT /identity/v3/projects/d1f75c2759794117a66f839dc2479912/users/7cde8c3305c84856835c9a7f11227157/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:32.036566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:32.064922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 895/1783] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:31 2022] PUT /identity/v3/projects/f516683cd3344c45adbdf92d448060c4/users/f80cbb2051af4224ad2a48c28ffde066/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:32.065928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:32.068878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:32.162366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aacbf-3e3b-4602-8d4b-93417bdfa9e4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:32.218153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebfcc5-3759-4e1d-895c-3ad597f686ca admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:32.224274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:32.225326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:32.225326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:32.232559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] PATH_INFO: `/v3/users/5900aaca6e5f47b196e5fb56e11b1521` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:32.236369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5900aaca6e5f47b196e5fb56e11b1521)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:32.239758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:32.260318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:32.312270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:32.312270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:32.312270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:32.312270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:32.313349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:32.314120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:32.318755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49f3e7-1cae-4b13-88d7-b1af44f78559 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:32.387659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 889/1784] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:32.387659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:32.387659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:32.387659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:32.434731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:32.436737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:32.470841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 890/1785] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:32.472576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:32.503382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199aa3a-cbea-423d-bdde-3682f63ab8ca None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:32.519562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:32.520287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:32.520578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:32.520956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:32.521728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:32.523374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:32.554518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 891/1786] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:32.558121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:32.558121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:32.589240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bd02e-6857-4568-a1a1-10b420fb1fe2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:32.689786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:32.689786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:32.689786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:32.689786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] PATH_INFO: `/v3/projects/259af367ae2f4919aeafb20d0a102d7f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:32.690157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259af367ae2f4919aeafb20d0a102d7f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:32.690157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:32.768738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:32.975610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259af367ae2f4919aeafb20d0a102d7f', 'request_id': 'req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:32.976338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5900aaca6e5f47b196e5fb56e11b1521'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:32.979969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5900aaca6e5f47b196e5fb56e11b1521'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:32.982523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5900aaca6e5f47b196e5fb56e11b1521'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:32.987036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259af367ae2f4919aeafb20d0a102d7f', 'request_id': 'req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:33.006842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5900aaca6e5f47b196e5fb56e11b1521'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:33.135118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259af367ae2f4919aeafb20d0a102d7f', 'request_id': 'req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:33.161573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] The token cache is being invalidate because project 259af367ae2f4919aeafb20d0a102d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:33.161573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:33.161573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:33.161573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 892/1787] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:32 2022] DELETE /identity/v3/projects/259af367ae2f4919aeafb20d0a102d7f => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:33.161573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:33.161900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.232216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cfca8-e6fc-40c3-8d2e-8df2efe55494 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:33.240825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:33.241595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.246021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.246021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.246021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:33.246021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:33.296775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5900aaca6e5f47b196e5fb56e11b1521'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:33.316539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 896/1788] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:32 2022] DELETE /identity/v3/users/5900aaca6e5f47b196e5fb56e11b1521 => generated 0 bytes in 1251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:33.317799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:33.323054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.334664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 893/1789] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:33 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:33.334664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:33.335529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.459403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26157071-a34a-4159-b446-8c7598b75886 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:33.464501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729adb3a-9239-42b0-8c31-88f88671964f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:33.481923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d215e1b-188c-49e4-ad81-a07a9b1d6aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:33.484647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d215e1b-188c-49e4-ad81-a07a9b1d6aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.486136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d215e1b-188c-49e4-ad81-a07a9b1d6aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.486432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d215e1b-188c-49e4-ad81-a07a9b1d6aee admin admin] PATH_INFO: `/v3/projects/f516683cd3344c45adbdf92d448060c4/users/f80cbb2051af4224ad2a48c28ffde066/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.493084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d215e1b-188c-49e4-ad81-a07a9b1d6aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f516683cd3344c45adbdf92d448060c4, user_id=f80cbb2051af4224ad2a48c28ffde066, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:33.495959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d215e1b-188c-49e4-ad81-a07a9b1d6aee admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:33.522357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 894/1790] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:33 2022] PUT /identity/v3/projects/f516683cd3344c45adbdf92d448060c4/users/f80cbb2051af4224ad2a48c28ffde066/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:33.524334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c670970-7329-4564-9885-668a397ffa8d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.524640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c670970-7329-4564-9885-668a397ffa8d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.525023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c670970-7329-4564-9885-668a397ffa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.565447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7c670970-7329-4564-9885-668a397ffa8d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:33.565447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7c670970-7329-4564-9885-668a397ffa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.565669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:33.565669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.565669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.565669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.577853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.603965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:33.606954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fd502-09d1-4672-b4ea-4637008492df service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:33.629958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 897/1791] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:33.632441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60f8a-bc11-4b50-91ad-503bbff470af None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.632660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60f8a-bc11-4b50-91ad-503bbff470af None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.632953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60f8a-bc11-4b50-91ad-503bbff470af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.673192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 895/1792] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:33.676164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-13f60f8a-bc11-4b50-91ad-503bbff470af None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:33.676787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c670970-7329-4564-9885-668a397ffa8d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:33.677781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-13f60f8a-bc11-4b50-91ad-503bbff470af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.690779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c670970-7329-4564-9885-668a397ffa8d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:33.697172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:33.698070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.698486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.698828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 898/1793] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:33.699218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.700044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:33.723780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70118e76-8af3-49ea-ae1a-614d753de61c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.723780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70118e76-8af3-49ea-ae1a-614d753de61c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.723780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70118e76-8af3-49ea-ae1a-614d753de61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.724175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:33.757596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-70118e76-8af3-49ea-ae1a-614d753de61c None None] MFA Rules not processed for user `f80cbb2051af4224ad2a48c28ffde066`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:33.758826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-70118e76-8af3-49ea-ae1a-614d753de61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.875318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 899/1794] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:33.877028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.877473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.877970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:33.908909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:33.910482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.920705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 900/1795] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:33.921622 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:33.923147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:33.991843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbfa38-cfc8-462f-b431-fba861ec83a0 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:33.995718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:33.996476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:33.996810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:33.997158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.008031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.009661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.023420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 901/1796] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:34.023994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.025362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:34.113503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f481e4e-5d3e-45e2-a720-7205a45d4dde service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.118416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.119063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.119351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.119630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.131743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.133620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.145813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 902/1797] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:34.146679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.164979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc222d31-29c8-41d2-aabd-55ea5dbf60f5 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.180758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.183326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.183326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.183326 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.188060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:34.188216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 896/1798] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:33 2022] POST /identity/v3/users => generated 287 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:34.188313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.188313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.189229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.209907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f0418-5af7-4167-8689-ea85b7e0cfee admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.218407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 903/1799] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:34.219459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.224519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.225193 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.225540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.225814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.226546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.229628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.233115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:34.237885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ba698-2db9-4985-93f0-e04089c119f6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.243439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.244082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.244371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.244650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.245424 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.247474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.264184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 897/1800] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:34.268275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.285961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d8517-c350-4dcd-b7eb-aa164c57e086 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.298281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.299270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.299516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.299821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.302279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.304499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.322473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 898/1801] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:34.327400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.339204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 904/1802] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:34 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:34.340023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d6838-be5a-422a-8ef8-33ef076d3efa admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.359233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.468164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.481465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.482267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.482759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] PATH_INFO: `/v3/projects/b9514d1fd0134e6587239c332adb4742/users/6bda96e73578467dab4985c7ec8d6cbf/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.534117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9514d1fd0134e6587239c332adb4742, user_id=6bda96e73578467dab4985c7ec8d6cbf, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.539895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.541357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa205aa0-664b-4191-8cda-364af4df7807 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.547026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57370824-8efc-4f98-b386-5939c4f15b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.547679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57370824-8efc-4f98-b386-5939c4f15b08 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.550846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57370824-8efc-4f98-b386-5939c4f15b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.550846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57370824-8efc-4f98-b386-5939c4f15b08 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.550846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57370824-8efc-4f98-b386-5939c4f15b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.551485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57370824-8efc-4f98-b386-5939c4f15b08 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.609045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 899/1803] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:34 2022] PUT /identity/v3/projects/b9514d1fd0134e6587239c332adb4742/users/6bda96e73578467dab4985c7ec8d6cbf/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:34.609045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.626893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:34.636839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 905/1804] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:34 2022] POST /identity/v3/users => generated 294 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:34.638738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.639044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.639522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.695287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] MFA Rules not processed for user `6bda96e73578467dab4985c7ec8d6cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:34.696813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:34.784127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80dbf3-3698-4bca-b3ff-44f432e37807 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:34.794666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:34.795421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:34.795707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:34.796031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:34.798921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:34.907589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:34.918790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 906/1805] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:34.918790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.939346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:34.942079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:34.975418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 900/1806] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:34.978727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:34.981951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:35.011169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbda39-00d6-4aea-9aaf-03a8f90c1882 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.018649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.019088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.019447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.019842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.021235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.024382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:35.038531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 907/1807] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:35.039429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:35.041426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:35.092908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac572-5435-4569-b97e-616dbcfe2904 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.099373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.100147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.100503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.100849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] PATH_INFO: `/v3/projects/b6398c32130945ed8d4f59d770b4b41c` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.102208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6398c32130945ed8d4f59d770b4b41c)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.112468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:35.180658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62490620-ab3d-411f-a058-c4b899c4d116 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.188007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.188884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.189273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.189628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.199804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.202172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:35.228664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 908/1808] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:35.253840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:35.253840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a68134-6ffc-472b-a13d-c3d1d7db9290 service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.256553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6398c32130945ed8d4f59d770b4b41c', 'request_id': 'req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:35.273161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6398c32130945ed8d4f59d770b4b41c', 'request_id': 'req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:35.387862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.388506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.388786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.389060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.389814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.391467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:35.405719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 909/1809] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:35.410056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:35.420882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30a20f-656e-4e83-bf58-053ed208164f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.428193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.428887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.429183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.430222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] PATH_INFO: `/v3/projects/b0b8e289621f40698bb95578ba7f496c/users/3f844893d1e0414da4e996d82f97e1f9/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.470765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b8e289621f40698bb95578ba7f496c, user_id=3f844893d1e0414da4e996d82f97e1f9, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.473990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:35.596745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6398c32130945ed8d4f59d770b4b41c', 'request_id': 'req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:35.613430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] The token cache is being invalidate because project b6398c32130945ed8d4f59d770b4b41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:35.614212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:35.615774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 910/1810] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:35 2022] PUT /identity/v3/projects/b0b8e289621f40698bb95578ba7f496c/users/3f844893d1e0414da4e996d82f97e1f9/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:35.616806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:35.618515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b9d14aa2-1b77-4cd4-9f0e-868627c08e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:35.620309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:35.623461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 901/1811] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:34 2022] DELETE /identity/v3/projects/b6398c32130945ed8d4f59d770b4b41c => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:35.626578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.627203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.628700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.688758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] MFA Rules not processed for user `3f844893d1e0414da4e996d82f97e1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:35.690726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:35.808240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f7bc-313f-47b8-97d3-b6c6346cf559 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.816721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 902/1812] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:35.816721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:35.816721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:35.817127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.817127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.817127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.817127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] PATH_INFO: `/v3/users/6bda96e73578467dab4985c7ec8d6cbf` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.819294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bda96e73578467dab4985c7ec8d6cbf)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.821287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:35.826983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b49d6-a461-45ae-8bb1-06654c02bc83 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:35.831791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:35.832499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:35.832766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:35.833126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] PATH_INFO: `/v3/users/aafc8bbb125742f08e1d533435b2f66b` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:35.834654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafc8bbb125742f08e1d533435b2f66b)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:35.836615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.052178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bda96e73578467dab4985c7ec8d6cbf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.053104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bda96e73578467dab4985c7ec8d6cbf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.053855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bda96e73578467dab4985c7ec8d6cbf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.072139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafc8bbb125742f08e1d533435b2f66b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.085929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bda96e73578467dab4985c7ec8d6cbf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.094411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafc8bbb125742f08e1d533435b2f66b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.095430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafc8bbb125742f08e1d533435b2f66b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.116198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafc8bbb125742f08e1d533435b2f66b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.203546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bda96e73578467dab4985c7ec8d6cbf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.221152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 911/1813] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:35 2022] DELETE /identity/v3/users/6bda96e73578467dab4985c7ec8d6cbf => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:36.221152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.221152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.270265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e7329ed5-7c0d-4615-bc65-4c0bcf116b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafc8bbb125742f08e1d533435b2f66b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:36.277225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 903/1814] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:35 2022] DELETE /identity/v3/users/aafc8bbb125742f08e1d533435b2f66b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:36.278599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0c818-3275-4852-abc4-ba1200e96ec0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.279067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0c818-3275-4852-abc4-ba1200e96ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.279593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0c818-3275-4852-abc4-ba1200e96ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.283313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:36.305437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef5a5f-6831-4c22-851a-25d303b021f9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.335313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-37c0c818-3275-4852-abc4-ba1200e96ec0 None None] MFA Rules not processed for user `7cde8c3305c84856835c9a7f11227157`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:36.337079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-37c0c818-3275-4852-abc4-ba1200e96ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.380073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.380073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.380073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.380073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.380529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.380529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.432038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 904/1815] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:36.433308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.433599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.433874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.453800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:36.455035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.465241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 905/1816] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:36.465241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.465867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.479385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e37e8-7b44-4f9a-8039-b132fd5bdbec None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.490806 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.491552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.491919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.492292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.493101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.495119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.511911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 912/1817] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:36 2022] POST /identity/v3/projects => generated 387 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:36.512762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.514976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:36.516858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.600671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a259f-2f59-4848-a280-7aad35c0b5d0 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.604943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.605138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.605467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.605745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.666701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 906/1818] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:36 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:36.674752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.677221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.679183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.681389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.701109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 913/1819] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:36.702314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.713756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ee9dc-7ea7-44fd-9def-79c4a943ff49 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.764126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d2014-ae31-4689-b420-d895a54719d5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.772172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.773136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.773284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.773497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.774201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.776197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.792044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.798863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.798863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.798863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.798863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.798863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.877501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 907/1820] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:36 2022] POST /identity/v3/users => generated 300 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:36.878724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.878996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 914/1821] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:36 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:36.881824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:36.882755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.902258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337343ac-2d4b-4b22-b6ad-52b445d5ee82 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.911356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5209a-c92a-4219-b0d8-61dee91038f1 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.915996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.917110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.917476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.918645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.920061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.922442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.929464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.929464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.929464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.929464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] PATH_INFO: `/v3/projects/789f1120bc59404c87a0973a92a39fe3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.929965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=789f1120bc59404c87a0973a92a39fe3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.929965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.939976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 915/1822] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:36.939976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:36.950093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff02ca-27ba-41e7-805a-185a17d5b20e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:36.957074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:36.958743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:36.959088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:36.959369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:36.961753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:36.965638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:36.981285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 916/1823] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:36.983193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.010781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804dedc-fbf3-478f-ae5b-bebfff5f77c5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.020948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.021678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.021967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.022253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] PATH_INFO: `/v3/projects/44a76dce2bbb4f3baafaaaa117615919/users/41270e7b9e8a4e039dcac3c6ff00038d/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.042119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a76dce2bbb4f3baafaaaa117615919, user_id=41270e7b9e8a4e039dcac3c6ff00038d, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.045038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.124023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 917/1824] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:36 2022] PUT /identity/v3/projects/44a76dce2bbb4f3baafaaaa117615919/users/41270e7b9e8a4e039dcac3c6ff00038d/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:37.125054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.130012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '789f1120bc59404c87a0973a92a39fe3', 'request_id': 'req-b7c22854-72f6-4dfd-b152-fb8df45a0c71'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.151091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '789f1120bc59404c87a0973a92a39fe3', 'request_id': 'req-b7c22854-72f6-4dfd-b152-fb8df45a0c71'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.206309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71962f-55b3-4a39-bf50-a4eaabc5acd6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.258468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '789f1120bc59404c87a0973a92a39fe3', 'request_id': 'req-b7c22854-72f6-4dfd-b152-fb8df45a0c71'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.283689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] The token cache is being invalidate because project 789f1120bc59404c87a0973a92a39fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:37.283689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.283689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.286936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 908/1825] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:36 2022] DELETE /identity/v3/projects/789f1120bc59404c87a0973a92a39fe3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:37.289229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.292127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.308613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7914-d151-4f58-9031-2da4f3f97074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.308978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7914-d151-4f58-9031-2da4f3f97074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.313359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7914-d151-4f58-9031-2da4f3f97074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.313359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7914-d151-4f58-9031-2da4f3f97074 admin admin] PATH_INFO: `/v3/projects/f6050f07434348caad5bfdc7e38f5a05/users/6a500036d0284c94b0e367363cab8e64/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.326743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7914-d151-4f58-9031-2da4f3f97074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6050f07434348caad5bfdc7e38f5a05, user_id=6a500036d0284c94b0e367363cab8e64, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.330072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7914-d151-4f58-9031-2da4f3f97074 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.371794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c22854-72f6-4dfd-b152-fb8df45a0c71 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.376529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.377155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.377465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.377740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.378445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.380696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.391244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 909/1826] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:37.392065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.393797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.407053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f100f-61c8-47c0-804e-5522734d24a4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.411895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.412514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.412797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.413069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] PATH_INFO: `/v3/projects/b9514d1fd0134e6587239c332adb4742` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.414258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9514d1fd0134e6587239c332adb4742)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.416188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.430740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 918/1827] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:37 2022] PUT /identity/v3/projects/f6050f07434348caad5bfdc7e38f5a05/users/6a500036d0284c94b0e367363cab8e64/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:37.433568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.433568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.433568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.452546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:37.453781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.519051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 919/1828] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:37.519051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.524148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9160f-642c-40b2-a418-13ab1d8f8d77 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.537150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9514d1fd0134e6587239c332adb4742', 'request_id': 'req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.544017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9514d1fd0134e6587239c332adb4742', 'request_id': 'req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.581659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.582259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.582545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.582836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] PATH_INFO: `/v3/projects/44a76dce2bbb4f3baafaaaa117615919/users/41270e7b9e8a4e039dcac3c6ff00038d/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.584988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a76dce2bbb4f3baafaaaa117615919, user_id=41270e7b9e8a4e039dcac3c6ff00038d, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.588457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.593492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:37.602000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 920/1829] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:37 2022] PUT /identity/v3/projects/44a76dce2bbb4f3baafaaaa117615919/users/41270e7b9e8a4e039dcac3c6ff00038d/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:37.603621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.604526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.613281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9514d1fd0134e6587239c332adb4742', 'request_id': 'req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.628496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] The token cache is being invalidate because project b9514d1fd0134e6587239c332adb4742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:37.629061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.630607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a7d4e19-caa0-43a8-8811-9dcb24a507d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:37.632671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 910/1830] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:37 2022] DELETE /identity/v3/projects/b9514d1fd0134e6587239c332adb4742 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:37.634703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957857-0d4f-483a-b390-05a92b966029 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.635412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957857-0d4f-483a-b390-05a92b966029 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.635412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957857-0d4f-483a-b390-05a92b966029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.678483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-99957857-0d4f-483a-b390-05a92b966029 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:37.688753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-99957857-0d4f-483a-b390-05a92b966029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.713637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01104dcd-ddc9-4fb6-b4fb-b0fca5594b82 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.724917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f0266-3be7-47b2-866a-c7b3a95ec24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.724917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f0266-3be7-47b2-866a-c7b3a95ec24e admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.724917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f0266-3be7-47b2-866a-c7b3a95ec24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.725242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f0266-3be7-47b2-866a-c7b3a95ec24e admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.725793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f0266-3be7-47b2-866a-c7b3a95ec24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.727435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6f0266-3be7-47b2-866a-c7b3a95ec24e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.759398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 921/1831] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:37.759398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.759398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.761381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.805345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:37.805345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.813444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 911/1832] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:37.813444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957857-0d4f-483a-b390-05a92b966029 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.813674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-99957857-0d4f-483a-b390-05a92b966029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.822570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 922/1833] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:37.829257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.835091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957857-0d4f-483a-b390-05a92b966029 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.837336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f8e65-7bd5-4bda-82d5-c9c114f06b0d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.841442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.842054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.842541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.842802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.843092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.843338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.843454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.844073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:37.844368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.844873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] PATH_INFO: `/v3/projects/f6050f07434348caad5bfdc7e38f5a05/users/6a500036d0284c94b0e367363cab8e64/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.846020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c87d4-ce5b-4738-8239-8136cfbaedbb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.848633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6050f07434348caad5bfdc7e38f5a05, user_id=6a500036d0284c94b0e367363cab8e64, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.851755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.866506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 923/1834] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:37.866506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3192ae-c2c8-4c13-845b-39f76485bdd5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.866506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3192ae-c2c8-4c13-845b-39f76485bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.866506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3192ae-c2c8-4c13-845b-39f76485bdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.873649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 912/1835] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:37 2022] PUT /identity/v3/projects/f6050f07434348caad5bfdc7e38f5a05/users/6a500036d0284c94b0e367363cab8e64/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:37.874669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:37.887081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4c3192ae-c2c8-4c13-845b-39f76485bdd5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:37.888284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4c3192ae-c2c8-4c13-845b-39f76485bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:37.951785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562c56e-691d-421d-8d44-8c998de2c1b1 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:37.957124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:37.958409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.958704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.958978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:37.959791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:37.960723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 924/1836] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:37.962549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:37.969608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:37.970238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:37.970624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.009578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] MFA Rules not processed for user `6a500036d0284c94b0e367363cab8e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:38.010857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:38.013420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:38.080072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 913/1837] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:37 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:38.081518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.090447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 925/1838] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:38.091763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.103013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10f1a3-e69f-4fe5-b71b-5646167315a1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.166552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11676515-c0f4-4f4b-8a3f-a64a467c43e0 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.178101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.179443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.180856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.182627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.184769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:38.185612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.186392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.188106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.188496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.188838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.189167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.194977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.194977 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.207896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 914/1839] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:38.209577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.212255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:38.271769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b91f3e-60d4-43a2-a9ba-7f1adcf55f29 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.276128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.276696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.276994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.277295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.339917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 926/1840] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:38 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:38.340805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.350837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.357462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb81a2-b0f2-413c-a1e2-c61f240dd9ad admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.357878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.367278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 915/1841] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:38.368243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.369735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.371588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.372125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.372643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.373827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.377765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.380350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3a58d-1e43-44d6-81f0-866d252f2335 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.386432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.387071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.387406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.387699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.388474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.390389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.406065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 916/1842] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:38.406919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.418884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d5ec-59f1-43e4-a5ed-e3c3c555bc81 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.423689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.424327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.424661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.424938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] PATH_INFO: `/v3/users/3f844893d1e0414da4e996d82f97e1f9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.426841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f844893d1e0414da4e996d82f97e1f9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.428781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.475001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 927/1843] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:38 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:38.476305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.545960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af62ea-e304-4974-bfbd-aa38d43a8d9b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.550896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.551546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.551831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.552133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.552859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.554556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.564649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 928/1844] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:38.565291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.575354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387c5d7-eae4-47a7-8097-b16e24eff792 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.580051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.580654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.580935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.581226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.581921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.584637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.604401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f844893d1e0414da4e996d82f97e1f9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:38.611588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f844893d1e0414da4e996d82f97e1f9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:38.612520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f844893d1e0414da4e996d82f97e1f9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:38.624448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f844893d1e0414da4e996d82f97e1f9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:38.716702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f844893d1e0414da4e996d82f97e1f9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:38.726438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 929/1845] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:38 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:38.727618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.730940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 917/1846] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:38 2022] DELETE /identity/v3/users/3f844893d1e0414da4e996d82f97e1f9 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:38.731458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:38.732393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.733830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:38.750119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630599e-0291-4734-98e7-188ea7ef0b13 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.811823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.813136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.813607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.813925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233dad0-70f4-46be-a42a-2f8705565418 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.814032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] PATH_INFO: `/v3/projects/add21697989041f4a690b2a6b7303fb5/users/7d478c4894c14ae8988ffee14faac23f/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.822774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b2809-3004-4160-aa24-de007e20e432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.822774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2809-3004-4160-aa24-de007e20e432 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.822774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2809-3004-4160-aa24-de007e20e432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.822774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b2809-3004-4160-aa24-de007e20e432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.823488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b2809-3004-4160-aa24-de007e20e432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.825106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b2809-3004-4160-aa24-de007e20e432 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.834190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] RBAC: Authorizing `identity:create_grant(project_id=add21697989041f4a690b2a6b7303fb5, user_id=7d478c4894c14ae8988ffee14faac23f, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.834190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.836252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 918/1847] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:38.837523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.838041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.838041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.859260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:38.859260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:38.870094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 919/1848] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:38.870954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.884333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae7c8-69f1-4738-ac1d-c8d424c8f6ec None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:38.889400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:38.890017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:38.890299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:38.890567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:38.891335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:38.892934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:38.907212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 920/1849] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:38.907212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:38.907212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:38.930106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 930/1850] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:38 2022] PUT /identity/v3/projects/add21697989041f4a690b2a6b7303fb5/users/7d478c4894c14ae8988ffee14faac23f/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:38.930952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.010408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf9756-c27f-47c3-8902-c3668c87d348 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.081360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147fc16-0ae5-46c8-b642-07e9ee6c540f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.082108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b4a69-911a-4c39-ae95-d08876050b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.083768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b4a69-911a-4c39-ae95-d08876050b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.084168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b4a69-911a-4c39-ae95-d08876050b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.085035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b4a69-911a-4c39-ae95-d08876050b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.088939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:39.092560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3f197-3f0c-4780-9fb4-29ada79248d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.093513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3f197-3f0c-4780-9fb4-29ada79248d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.094297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3f197-3f0c-4780-9fb4-29ada79248d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.094590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3f197-3f0c-4780-9fb4-29ada79248d3 admin admin] PATH_INFO: `/v3/projects/31d12fe3fd17458cb79ae88b77fb26e7/users/27bbbe7f97104ead93705e2fc9283083/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.101346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b4a69-911a-4c39-ae95-d08876050b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.102800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b4a69-911a-4c39-ae95-d08876050b16 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.118163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3f197-3f0c-4780-9fb4-29ada79248d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d12fe3fd17458cb79ae88b77fb26e7, user_id=27bbbe7f97104ead93705e2fc9283083, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.119540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 921/1851] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:39.120725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49212ec-b895-4836-9511-bbcaf5e731a1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.121016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49212ec-b895-4836-9511-bbcaf5e731a1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.121180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3f197-3f0c-4780-9fb4-29ada79248d3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.121293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49212ec-b895-4836-9511-bbcaf5e731a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.153177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c49212ec-b895-4836-9511-bbcaf5e731a1 None None] MFA Rules not processed for user `7d478c4894c14ae8988ffee14faac23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:39.155026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c49212ec-b895-4836-9511-bbcaf5e731a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:39.190791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 931/1852] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:38 2022] PUT /identity/v3/projects/31d12fe3fd17458cb79ae88b77fb26e7/users/27bbbe7f97104ead93705e2fc9283083/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:39.202236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.202571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.202846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.237939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] MFA Rules not processed for user `41270e7b9e8a4e039dcac3c6ff00038d`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:39.239763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:39.291564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 922/1853] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:39.293322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.293682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.293959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.329770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] MFA Rules not processed for user `27bbbe7f97104ead93705e2fc9283083`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:39.330049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:39.333996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 932/1854] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:39.335054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.352652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:39.413443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f13a76-279c-4b8e-a8f3-5e0fd0df19f1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.423517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.423517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.423517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.423517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.424131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.424577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.451656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 923/1855] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:39.455994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.457994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:39.533050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5084448-64eb-4adf-8a2c-a5cf31e7e769 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.533292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 933/1856] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:39.534172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.599780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.600917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.601234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.601502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.603338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:39.631194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff6254-7416-4c89-a215-9bc89e323c51 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.649264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.651257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.651386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.651632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.662260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.664293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.677928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:39.700890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.702382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.712665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 924/1857] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:39.713502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.725564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7c9c0-b609-4d08-ae17-2cf2f3eae157 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.730518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.731972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.731972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.731972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.741585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.741585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.818165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 934/1858] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/projects => generated 367 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:39.819699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.823143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:39.921601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 925/1859] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:39.931859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:39.935922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18617268-84d1-4843-a83c-152b637899f2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.944769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.945911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.946261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.946641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] PATH_INFO: `/v3/projects/b0b8e289621f40698bb95578ba7f496c` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.951924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b8e289621f40698bb95578ba7f496c)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.954218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:39.959635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8320b-a7e7-412e-af16-4aff0719da40 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:39.968942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:39.969763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:39.970093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:39.970610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:39.971300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:39.979776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.056033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 926/1860] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:39 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:40.057461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.069561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85526f34-05cd-4870-90c5-d0a774a5c74a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.075668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.076426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.076780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.078769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.079565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.082825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.094804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 927/1861] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:40.096381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.111049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabce18-8bf8-418f-b065-872890360303 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.121005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.121798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.122075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.122349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.123084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.124815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.135648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b8e289621f40698bb95578ba7f496c', 'request_id': 'req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.144933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 928/1862] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:40.144933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.146496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b8e289621f40698bb95578ba7f496c', 'request_id': 'req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.159018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4954e86-7bd5-43af-b8c8-cf7dfa3083bd admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.241503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.242210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.242490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.242764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] PATH_INFO: `/v3/projects/2fe3b738afe64bd296bd76c178f139c3/users/891f82868aaa4f8997b1e1f60a7fa9d8/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.247441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b8e289621f40698bb95578ba7f496c', 'request_id': 'req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.263770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe3b738afe64bd296bd76c178f139c3, user_id=891f82868aaa4f8997b1e1f60a7fa9d8, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.264617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] The token cache is being invalidate because project b0b8e289621f40698bb95578ba7f496c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:40.265056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.266554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.267352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.270642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 935/1863] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:39 2022] DELETE /identity/v3/projects/b0b8e289621f40698bb95578ba7f496c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:40.271899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.278539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:40.294058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd8f0c-94f1-424a-8e28-1eecb1b9abcf admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.299789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.300491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.300774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.301794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] PATH_INFO: `/v3/projects/65b13e572a9449e49c4176b52e4ae711/users/cf142a8a66464c1dbe21dabdd696ed39/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.321889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b13e572a9449e49c4176b52e4ae711, user_id=cf142a8a66464c1dbe21dabdd696ed39, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.325704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.346391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 929/1864] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:40 2022] PUT /identity/v3/projects/2fe3b738afe64bd296bd76c178f139c3/users/891f82868aaa4f8997b1e1f60a7fa9d8/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:40.347016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.349414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:40.401116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 936/1865] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:40 2022] PUT /identity/v3/projects/65b13e572a9449e49c4176b52e4ae711/users/cf142a8a66464c1dbe21dabdd696ed39/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:40.403460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.405453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:40.453515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097197c5-b002-483e-92e2-d2dda514b272 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.548516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f79786-f3b1-4dec-a1e8-0411fb9ad693 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.562265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.563084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.563804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.564301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.564647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.565543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.567195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.569543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.569753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.570111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] PATH_INFO: `/v3/users/6a500036d0284c94b0e367363cab8e64` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.572645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a500036d0284c94b0e367363cab8e64)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.574463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.593398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 937/1866] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:40.594361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.616721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dde9be-0a35-4896-997f-b3bfe9f21bdc admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.636790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.637609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.637895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.638170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.638879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.640535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.652496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 938/1867] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:40.653593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.665239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154f4f4-90a1-442b-9851-08f41c000e66 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.670286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.671016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.671165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.671440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] PATH_INFO: `/v3/projects/2fe3b738afe64bd296bd76c178f139c3/users/891f82868aaa4f8997b1e1f60a7fa9d8/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.673585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe3b738afe64bd296bd76c178f139c3, user_id=891f82868aaa4f8997b1e1f60a7fa9d8, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.676468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.688699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 939/1868] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:40 2022] PUT /identity/v3/projects/2fe3b738afe64bd296bd76c178f139c3/users/891f82868aaa4f8997b1e1f60a7fa9d8/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:40.693825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.764488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b7c43-8e8f-470c-966a-bcdc5a1c1cb9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:40.771555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44cb4f-b4eb-4581-9eb7-4148bf6dada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:40.772266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44cb4f-b4eb-4581-9eb7-4148bf6dada8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.772547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44cb4f-b4eb-4581-9eb7-4148bf6dada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.772823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44cb4f-b4eb-4581-9eb7-4148bf6dada8 admin admin] PATH_INFO: `/v3/projects/65b13e572a9449e49c4176b52e4ae711/users/cf142a8a66464c1dbe21dabdd696ed39/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.774852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44cb4f-b4eb-4581-9eb7-4148bf6dada8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b13e572a9449e49c4176b52e4ae711, user_id=cf142a8a66464c1dbe21dabdd696ed39, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:40.777688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44cb4f-b4eb-4581-9eb7-4148bf6dada8 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:40.789894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 940/1869] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:40 2022] PUT /identity/v3/projects/65b13e572a9449e49c4176b52e4ae711/users/cf142a8a66464c1dbe21dabdd696ed39/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:40.791298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b23f3c-fed2-4a9e-8011-d921c9303d6b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.791657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b23f3c-fed2-4a9e-8011-d921c9303d6b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.792004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b23f3c-fed2-4a9e-8011-d921c9303d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.827525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-d2b23f3c-fed2-4a9e-8011-d921c9303d6b None None] MFA Rules not processed for user `891f82868aaa4f8997b1e1f60a7fa9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:40.829153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d2b23f3c-fed2-4a9e-8011-d921c9303d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:40.845611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a500036d0284c94b0e367363cab8e64'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.854204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a500036d0284c94b0e367363cab8e64'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.857601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a500036d0284c94b0e367363cab8e64'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.871352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a500036d0284c94b0e367363cab8e64'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.919009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 941/1870] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:40.920446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:40.920683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:40.920952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:40.950766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a500036d0284c94b0e367363cab8e64'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:40.964337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 930/1871] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:40 2022] DELETE /identity/v3/users/6a500036d0284c94b0e367363cab8e64 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:40.964337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:40.964337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:40.985929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] MFA Rules not processed for user `cf142a8a66464c1dbe21dabdd696ed39`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:40.989546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.073404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8e831-9145-4502-8dd5-540961d3b255 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:41.079734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:41.080855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.081625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.083139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.086420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.143828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 942/1872] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:41.154867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:41.156974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.181600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25902a-a1e9-40f4-9beb-30ce7e753ad7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:41.185646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:41.186278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.186560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.191338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.198453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:41.200656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:41.201921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:41.203553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:41.214393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 931/1873] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:41.218028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 943/1874] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:39:41.231580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:41.333157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:41.342563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429a56a-fb27-44c4-a2a0-6f5ee8ae1462 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:41.346869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:41.347564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.348693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.349074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.351429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.413025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:41.414503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97202d92-c395-4592-909d-b1a1245674df service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:41.422096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:41.423610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.424791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 932/1875] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:41.498207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f36891-824d-42fa-aed7-9620ab9a66f2 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:41.503957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:41.504671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.504959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.505261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] PATH_INFO: `/v3/projects/f6050f07434348caad5bfdc7e38f5a05` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.506430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6050f07434348caad5bfdc7e38f5a05)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:41.508509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:41.614455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6050f07434348caad5bfdc7e38f5a05', 'request_id': 'req-99765c80-65da-4ffa-9da3-827abc1e4e1c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:41.620667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6050f07434348caad5bfdc7e38f5a05', 'request_id': 'req-99765c80-65da-4ffa-9da3-827abc1e4e1c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:41.653846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:41.721876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6050f07434348caad5bfdc7e38f5a05', 'request_id': 'req-99765c80-65da-4ffa-9da3-827abc1e4e1c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:41.733252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] The token cache is being invalidate because project f6050f07434348caad5bfdc7e38f5a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:41.733545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:41.734209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-99765c80-65da-4ffa-9da3-827abc1e4e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:41.735744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 944/1876] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:41 2022] DELETE /identity/v3/projects/f6050f07434348caad5bfdc7e38f5a05 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:41.740915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.741219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.741516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.760975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:41.762207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.845339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 933/1877] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:41.854198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.854487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.854787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.890222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:41.891515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:41.902879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 945/1878] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:41.906413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:41.918453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5c20c-3280-48ae-8384-4cd120b1a63f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:41.930940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:41.931725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.932103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.932425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.933307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:41.933665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:41.936931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:41.961319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 934/1879] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:41.965437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:41.975647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37eae12-8f0a-4c6d-a87f-c016cdd6ac90 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:41.980444 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:41.981067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:41.981388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:41.981665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:41.982348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:41.984270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.101837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 946/1880] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:41 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:42.112597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.136606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32480d08-1ed1-4a4e-8ffc-a756dce8a5ec admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.193461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.193461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.193622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.193828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.194549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.196699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.298970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 935/1881] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:42 2022] POST /identity/v3/users => generated 291 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:42.303142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.313449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f72049-2648-4684-9ce7-5b0c14131fc2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.318215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.318821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.319106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.319381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.320248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.321752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.331666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 947/1882] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:42.336434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.346666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d25951-4103-4168-9021-ad5f83cd49d2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.352247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122812f0-e9f9-477f-98f5-fbe7f8ddcca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.352865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122812f0-e9f9-477f-98f5-fbe7f8ddcca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.353146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122812f0-e9f9-477f-98f5-fbe7f8ddcca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.353454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122812f0-e9f9-477f-98f5-fbe7f8ddcca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.354188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122812f0-e9f9-477f-98f5-fbe7f8ddcca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.355801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122812f0-e9f9-477f-98f5-fbe7f8ddcca5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.371092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 936/1883] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:42.371399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.381837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54fb87-fb67-4ffd-99f4-81e93d6f8738 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.386384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.387038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.387321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.387598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] PATH_INFO: `/v3/projects/5205e6e7a5514213a770c39cbc807456/users/26cf6bf107264582a236ceaabf58aaed/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.402558 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5205e6e7a5514213a770c39cbc807456, user_id=26cf6bf107264582a236ceaabf58aaed, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.405245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.493924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 948/1884] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:42 2022] PUT /identity/v3/projects/5205e6e7a5514213a770c39cbc807456/users/26cf6bf107264582a236ceaabf58aaed/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:42.504759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.505052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.505381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.533152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] MFA Rules not processed for user `26cf6bf107264582a236ceaabf58aaed`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:42.534381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:42.598936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 937/1885] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:42.603637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.664675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba526-262d-4128-842d-f1afa08c6d84 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.669865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.670491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.671807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:42.672251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.672581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.673315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.678321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.768079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 949/1886] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:42 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:42.772212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.783081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f55c5b-7f57-408b-a864-37740cf17d60 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.837189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:42.841473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.842279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.842688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.843467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.844153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.846512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.911074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 938/1887] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:42 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:42.915174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.927396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940a30-9b63-458c-8fc5-dac834158ccd admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.932073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.932785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.933064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.933366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.934058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.935690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.948803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 950/1888] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:42.953009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:42.963829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8173f9-39c8-4d47-9d57-07d9d3818006 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:42.968741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:42.969474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:42.969750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:42.970019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] PATH_INFO: `/v3/projects/913582e9a89948c4873900f92f5f8d68/users/65c0ee66611f443abfec6cc97b5bbc8a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:42.985410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913582e9a89948c4873900f92f5f8d68, user_id=65c0ee66611f443abfec6cc97b5bbc8a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:42.988912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:42.991284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.041772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 939/1889] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:42 2022] PUT /identity/v3/projects/913582e9a89948c4873900f92f5f8d68/users/65c0ee66611f443abfec6cc97b5bbc8a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:43.045731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:43.055830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c892f-6772-4a8f-b962-1ee4fc1b81ba admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:43.113608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742273-66d4-4923-8d31-97f6045ed64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:43.114267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742273-66d4-4923-8d31-97f6045ed64a admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:43.114555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742273-66d4-4923-8d31-97f6045ed64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:43.114829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742273-66d4-4923-8d31-97f6045ed64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:43.115529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742273-66d4-4923-8d31-97f6045ed64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:43.117114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742273-66d4-4923-8d31-97f6045ed64a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:43.127743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 951/1890] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:43.132073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:43.142386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42176e9c-2833-4f08-8aae-7cf1a5b96532 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:43.147582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:43.148251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:43.148538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:43.148810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] PATH_INFO: `/v3/projects/913582e9a89948c4873900f92f5f8d68/users/65c0ee66611f443abfec6cc97b5bbc8a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:43.150850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913582e9a89948c4873900f92f5f8d68, user_id=65c0ee66611f443abfec6cc97b5bbc8a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:43.153548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:43.168660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 940/1891] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:43 2022] PUT /identity/v3/projects/913582e9a89948c4873900f92f5f8d68/users/65c0ee66611f443abfec6cc97b5bbc8a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:43.179526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.179888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:43.180274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:43.180625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:43.202898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.214460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] MFA Rules not processed for user `65c0ee66611f443abfec6cc97b5bbc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:43.215709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:43.302117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 952/1892] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:43.313938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:43.315849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:43.334353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.380321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81568492-f8a6-4cee-b650-8542001604d2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:43.383901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.385588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:43.385588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:43.385978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:43.386068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:43.396015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:43.397685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:43.408357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 941/1893] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:43.613715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.773731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:43.969125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:44.005177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:44.197446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:44.478217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:44.495046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc53e13-a6b3-4990-b16b-67192af056ad None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:44.495548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c0ee66611f443abfec6cc97b5bbc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '913582e9a89948c4873900f92f5f8d68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:44.496418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:44.496714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:44.496997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:44.497740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:44.499419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:44.501130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:44.517573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 953/1894] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:44.522239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:44.532224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dbc2a-115a-431c-b735-b54503442159 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:44.550859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c0ee66611f443abfec6cc97b5bbc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '913582e9a89948c4873900f92f5f8d68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:44.551794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:44.552146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:44.552508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:44.554203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:44.556969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:44.559292 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:44.631972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:44.640424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 942/1895] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:44 2022] POST /identity/v3/projects => generated 353 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:44.645193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:44.655441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b84e-5cf4-432b-be1d-3fd55e65188a tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:44.724486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c0ee66611f443abfec6cc97b5bbc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '913582e9a89948c4873900f92f5f8d68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:44.725116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:44.726142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:44.726142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:44.726398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:44.726710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:44.728339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:44.741107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 954/1896] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:39:44.787392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:44.798126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003518f-86a9-4c1b-b94b-034065b3f8d2 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:44.803519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '65c0ee66611f443abfec6cc97b5bbc8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '913582e9a89948c4873900f92f5f8d68', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:44.804176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:44.804462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:44.804736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] PATH_INFO: `/v3/projects/05d5d2bdcb8740a79a118d984f8e8aee` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:44.806410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorizing `identity:delete_project(project_id=05d5d2bdcb8740a79a118d984f8e8aee)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:44.808605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:44.912462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d5d2bdcb8740a79a118d984f8e8aee', 'request_id': 'req-542e995d-508d-40a2-92c0-96ac739d69c6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:44.919579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d5d2bdcb8740a79a118d984f8e8aee', 'request_id': 'req-542e995d-508d-40a2-92c0-96ac739d69c6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:44.956315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:44.973085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d5d2bdcb8740a79a118d984f8e8aee', 'request_id': 'req-542e995d-508d-40a2-92c0-96ac739d69c6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:44.984485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] The token cache is being invalidate because project 05d5d2bdcb8740a79a118d984f8e8aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:44.987190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:44.987190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:44.995084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 943/1897] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:44 2022] DELETE /identity/v3/projects/05d5d2bdcb8740a79a118d984f8e8aee => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:45.562150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:46.025039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:46.066766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:46.068600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:46.127890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fca375-7969-43c3-b401-de1930a87746 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:46.173945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:46.179395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:46.180025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:46.180311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:46.180585 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:46.182396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:46.251356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:46.252812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa54b6-2cab-4fcc-b681-fa33fa2107cf service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:46.262742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 955/1898] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:46.614133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:46.647870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:46.651160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:46.718018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e995d-508d-40a2-92c0-96ac739d69c6 tempest-VolumeQuotaClassesTest-1571485622 tempest-VolumeQuotaClassesTest-1571485622-project-admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:46.722609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:46.723428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:46.723711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:46.723991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] PATH_INFO: `/v3/users/26cf6bf107264582a236ceaabf58aaed` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:46.725157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cf6bf107264582a236ceaabf58aaed)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:46.727165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:46.980054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:46.982703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cf6bf107264582a236ceaabf58aaed'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:46.990016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cf6bf107264582a236ceaabf58aaed'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:46.990932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cf6bf107264582a236ceaabf58aaed'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.005186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cf6bf107264582a236ceaabf58aaed'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.059780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cf6bf107264582a236ceaabf58aaed'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.070457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 944/1899] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:46 2022] DELETE /identity/v3/users/26cf6bf107264582a236ceaabf58aaed => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:47.075014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:47.075481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:47.075572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:47.096590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:47.098380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:47.180360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 956/1900] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:47.188265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:47.189722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:47.267474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f3bf1-af31-449f-8fc2-569990f65597 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:47.273429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:47.273429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:47.273429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:47.273429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:47.289774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:47.291571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:47.308109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 945/1901] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:47.746711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:47.748227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:47.761594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479dace-4df5-4458-a25e-0bb8d60dcfe5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:47.766512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:47.767153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:47.767436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:47.767741 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] PATH_INFO: `/v3/projects/5205e6e7a5514213a770c39cbc807456` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:47.769329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5205e6e7a5514213a770c39cbc807456)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:47.771683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:47.904810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5205e6e7a5514213a770c39cbc807456', 'request_id': 'req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.911643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5205e6e7a5514213a770c39cbc807456', 'request_id': 'req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.979036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5205e6e7a5514213a770c39cbc807456', 'request_id': 'req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.981052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:47.983179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:47.993031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] The token cache is being invalidate because project 5205e6e7a5514213a770c39cbc807456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:47.993290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.994070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:47.996083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 957/1902] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:47 2022] DELETE /identity/v3/projects/5205e6e7a5514213a770c39cbc807456 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:47.999301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c870f-c4e3-4ed3-82b7-9550c932f3c3 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:47.999627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:48.001004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:48.027721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:48.066914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:48.067681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:48.068058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:48.068399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] PATH_INFO: `/v3/users/27bbbe7f97104ead93705e2fc9283083` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:48.070770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27bbbe7f97104ead93705e2fc9283083)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:48.073126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:48.076263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc67ed-467a-4fa0-96c1-356ef4bebcdd admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:48.081039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:48.081805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:48.083239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:48.083728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] PATH_INFO: `/v3/users/65c0ee66611f443abfec6cc97b5bbc8a` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:48.084987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c0ee66611f443abfec6cc97b5bbc8a)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:48.087083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:48.224276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27bbbe7f97104ead93705e2fc9283083'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.232648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27bbbe7f97104ead93705e2fc9283083'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.233426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27bbbe7f97104ead93705e2fc9283083'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.254491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:48.272472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27bbbe7f97104ead93705e2fc9283083'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.279283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c0ee66611f443abfec6cc97b5bbc8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.280339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c0ee66611f443abfec6cc97b5bbc8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.282483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c0ee66611f443abfec6cc97b5bbc8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.318174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c0ee66611f443abfec6cc97b5bbc8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.401901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-927f7aa4-7d2d-4314-b64c-d2f8012cfba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27bbbe7f97104ead93705e2fc9283083'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.408462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 946/1903] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:47 2022] DELETE /identity/v3/users/27bbbe7f97104ead93705e2fc9283083 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:48.413859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:48.414151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:48.414483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:48.434650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:48.436114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:48.470460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c0ee66611f443abfec6cc97b5bbc8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:48.491951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 958/1904] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:47 2022] DELETE /identity/v3/users/65c0ee66611f443abfec6cc97b5bbc8a => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:48.523711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 947/1905] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:48.530046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:48.531690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:48.592026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74a27-661c-43e1-83d3-d62c651783fe admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:48.595713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:48.598426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:48.598426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:48.598426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:48.650323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:48.661446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:48.663042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:48.673170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 959/1906] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:48.943872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:48.945305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:48.964975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb276a-0b5c-4a91-a8c2-c23f77094e10 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:48.965290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:48.965978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:48.966210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:48.966897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] PATH_INFO: `/v3/projects/913582e9a89948c4873900f92f5f8d68` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:48.967662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913582e9a89948c4873900f92f5f8d68)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:48.969709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.088306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913582e9a89948c4873900f92f5f8d68', 'request_id': 'req-d0772356-4de4-4ddf-af60-f29138451ee3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.094541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913582e9a89948c4873900f92f5f8d68', 'request_id': 'req-d0772356-4de4-4ddf-af60-f29138451ee3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.140906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.142203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:49.157592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7c8b8-2961-4570-8182-1cedf3b538ea service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.160505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913582e9a89948c4873900f92f5f8d68', 'request_id': 'req-d0772356-4de4-4ddf-af60-f29138451ee3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.181396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] The token cache is being invalidate because project 913582e9a89948c4873900f92f5f8d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:49.181691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.189656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d0772356-4de4-4ddf-af60-f29138451ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.195170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 948/1907] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:48 2022] DELETE /identity/v3/projects/913582e9a89948c4873900f92f5f8d68 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:49.196650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3482-2f8c-4ef9-a58f-5401eaf8a627 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.196937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3482-2f8c-4ef9-a58f-5401eaf8a627 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.197237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd3482-2f8c-4ef9-a58f-5401eaf8a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.230394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2cdd3482-2f8c-4ef9-a58f-5401eaf8a627 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:49.232473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2cdd3482-2f8c-4ef9-a58f-5401eaf8a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:49.245742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.250395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.250395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.250395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] PATH_INFO: `/v3/projects/31d12fe3fd17458cb79ae88b77fb26e7` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.250395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d12fe3fd17458cb79ae88b77fb26e7)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.252732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.302368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 949/1908] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:49.309944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.310228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.310501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.324969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:49.330994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:49.332922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:49.341154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 950/1909] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:49.345400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.354518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103aec3b-161a-4170-b91f-21e4e96078cc None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.359452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.360083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.360459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.360740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.361475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:49.361781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.363826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.375425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 951/1910] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:39:49.380837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.391123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a0e4f-e60a-4d20-bcc9-ad135e6f9210 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.397685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d12fe3fd17458cb79ae88b77fb26e7', 'request_id': 'req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.404251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d12fe3fd17458cb79ae88b77fb26e7', 'request_id': 'req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.450023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.450713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.451062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.451388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.452109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.454098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.470265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d12fe3fd17458cb79ae88b77fb26e7', 'request_id': 'req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.482264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] The token cache is being invalidate because project 31d12fe3fd17458cb79ae88b77fb26e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:49.482549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.483198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.484709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 960/1911] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:49 2022] DELETE /identity/v3/projects/31d12fe3fd17458cb79ae88b77fb26e7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:49.489658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.490942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:49.502985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d1c30-3c9b-4ac3-8a68-ef70baaa4635 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.507503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.508135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.508395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.508670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] PATH_INFO: `/v3/users/cf142a8a66464c1dbe21dabdd696ed39` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.509840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf142a8a66464c1dbe21dabdd696ed39)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.511770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.527242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 952/1912] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:49 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:49.528194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.529876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:49.600141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732501e-80f2-4a93-afd2-bf0dfb65dac7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.605523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.606163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.606493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.606734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.607418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.609537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.690115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf142a8a66464c1dbe21dabdd696ed39'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.690846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf142a8a66464c1dbe21dabdd696ed39'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.692255 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf142a8a66464c1dbe21dabdd696ed39'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.706278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf142a8a66464c1dbe21dabdd696ed39'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.707927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 953/1913] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:49 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:49.711640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.713793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:49.727473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e82e96-b75d-4c95-8982-afa81ad4706b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.732631 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.733286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.733572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.733879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.735121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.736797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.748334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 954/1914] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:49.752392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.763608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5e718-4cba-409b-9b36-a7b9d4d59812 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.768736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.769425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.769949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.770247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.770971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.772580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.788032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 955/1915] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:49.791274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:49.796119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-02f8d685-8ca8-4481-9ab6-1049caee9e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf142a8a66464c1dbe21dabdd696ed39'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:49.798515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:49.801603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ff818-cd9e-4356-91bb-abea89d31f2a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:49.806395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:49.811772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.811772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.811772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] PATH_INFO: `/v3/projects/52d97e020205403898d03abb282ec539/users/1ccdef4ac31b400c85b8b64c5a604e05/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.812112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 961/1916] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:39:49 2022] DELETE /identity/v3/users/cf142a8a66464c1dbe21dabdd696ed39 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:49.823743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d97e020205403898d03abb282ec539, user_id=1ccdef4ac31b400c85b8b64c5a604e05, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:49.826646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:49.916545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 956/1917] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:49 2022] PUT /identity/v3/projects/52d97e020205403898d03abb282ec539/users/1ccdef4ac31b400c85b8b64c5a604e05/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:49.927442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:49.927920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:49.928221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:49.955682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] MFA Rules not processed for user `1ccdef4ac31b400c85b8b64c5a604e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:49.956933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:50.023033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 962/1918] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:50.027458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.048463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:50.093059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e702c3-4488-4d13-af51-d2b4812f6d4f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.099460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.100084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.100363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.100634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.101398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.103391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.180645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 957/1919] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:50.190235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.196234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f84e3-dc8a-4cfd-9a97-2a942ffcf995 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.209398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.211218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:50.264627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.265278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.265563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.265835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.266655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.268438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.293927 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9d6b9-177a-4a61-9888-40937aaa8bea admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.298775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.299420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.299702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.299976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] PATH_INFO: `/v3/projects/65b13e572a9449e49c4176b52e4ae711` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.301919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b13e572a9449e49c4176b52e4ae711)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.303988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.339330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 963/1920] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:50.343232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.353480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfad70f-07db-473d-a799-ed911d4930ca admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.358217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.358859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.363760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.363760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.363760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.363760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.374810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 964/1921] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:50.380698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.391352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df92b1-44ab-4239-b765-655d631d582f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.396736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.397387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.397667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.397968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.398669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.400721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.416354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 965/1922] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:50.417764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.423162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b13e572a9449e49c4176b52e4ae711', 'request_id': 'req-3d05e261-c729-40f8-bba7-caba20eea8f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:50.430807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a5c6c-b99c-49e9-814c-fd995e5d4400 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.444244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b13e572a9449e49c4176b52e4ae711', 'request_id': 'req-3d05e261-c729-40f8-bba7-caba20eea8f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:50.490044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:50.502216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108d4ad-aed6-44bd-8821-a1124ae9a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.502925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108d4ad-aed6-44bd-8821-a1124ae9a56f admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.503209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108d4ad-aed6-44bd-8821-a1124ae9a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.503484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108d4ad-aed6-44bd-8821-a1124ae9a56f admin admin] PATH_INFO: `/v3/projects/55cb77d13eaf4a358768312054d223a5/users/d0121578fc3a42ba8213556293fd2b95/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.519201 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108d4ad-aed6-44bd-8821-a1124ae9a56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cb77d13eaf4a358768312054d223a5, user_id=d0121578fc3a42ba8213556293fd2b95, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.522053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108d4ad-aed6-44bd-8821-a1124ae9a56f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.532013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b13e572a9449e49c4176b52e4ae711', 'request_id': 'req-3d05e261-c729-40f8-bba7-caba20eea8f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:50.543658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] The token cache is being invalidate because project 65b13e572a9449e49c4176b52e4ae711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:39:50.544000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:50.544603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-3d05e261-c729-40f8-bba7-caba20eea8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:39:50.546274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 958/1923] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:39:50 2022] DELETE /identity/v3/projects/65b13e572a9449e49c4176b52e4ae711 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:50.552078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29a48f-47a6-4630-adb0-7c6bccf79ec7 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.552367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29a48f-47a6-4630-adb0-7c6bccf79ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.552639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29a48f-47a6-4630-adb0-7c6bccf79ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.572356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-dd29a48f-47a6-4630-adb0-7c6bccf79ec7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:50.573584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-dd29a48f-47a6-4630-adb0-7c6bccf79ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:50.581307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 959/1924] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:50.589175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.589483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.589756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.603766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 966/1925] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:50 2022] PUT /identity/v3/projects/55cb77d13eaf4a358768312054d223a5/users/d0121578fc3a42ba8213556293fd2b95/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:50.611233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:50.612717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:50.616250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.616653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.616942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.632913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:50.646617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] MFA Rules not processed for user `d0121578fc3a42ba8213556293fd2b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:50.647883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:50.679395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 960/1926] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:50.683699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.695621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de957d0-57ba-4ef5-b7da-877c6d37a36a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.701221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.706157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.706157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.706157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.706157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:39:50.706157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.706157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.718263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 961/1927] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:39:50.722492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.732433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dabceba-1068-4477-bcaf-11027a929111 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.732572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 967/1928] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:50.737494 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.737679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.738132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.738467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.738766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.739113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:50.739498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.741442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.752048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb87f2a-07fd-45e5-9cc5-508f561e41b4 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.757138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.757775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.758056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.758328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.758991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.760906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.833113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 962/1929] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:50.837135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.847020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 968/1930] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:50.848917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:50.923025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83369224-87ac-4f39-bf03-fcaaf7fb894d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.928603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.929285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.929574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.929854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.930521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.932787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:50.937267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef89766-1404-4f94-8865-3ad950a546e8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:50.944194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:50.945002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:50.945595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:50.946343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:50.946563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:50.949012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.000016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 969/1931] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:51.002459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.015078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea0e8f-920e-4c30-a53a-16ac01236fba admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.020506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.021181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.021505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.021815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.022513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.029390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.038270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 970/1932] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:51.039283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.050308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 963/1933] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:50 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:51.050606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34703b08-be2a-4a22-8129-8c1f01f2bed7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.053230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.056125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.056876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.057310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.057591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] PATH_INFO: `/v3/projects/a33d01a937774e999dd56d6a5560640f/users/8757ef30a353447b9059719a2cf6d474/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.063331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b761be-ba32-4c7e-b886-0de8452e32f5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.068163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.068752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.069031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.069346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.070038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.071651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.074990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33d01a937774e999dd56d6a5560640f, user_id=8757ef30a353447b9059719a2cf6d474, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.078782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.080540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:51.082770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 964/1934] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:51.086463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.096026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb3d3d-e41f-4b33-920e-b63b84dc57db admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.101471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.102120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.102401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.102675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.103361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.104943 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.115899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 965/1935] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:51.120029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.129452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045fea7-310d-4cf4-a315-e06f81544999 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.134069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd5d72-d040-4b9f-9f70-12151d205fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.134696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5d72-d040-4b9f-9f70-12151d205fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.134957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5d72-d040-4b9f-9f70-12151d205fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.135229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5d72-d040-4b9f-9f70-12151d205fd3 admin admin] PATH_INFO: `/v3/projects/8a3f8febaf594522aa46977d0e59e342/users/8a89865b46b94140bde14022f2a547e2/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.150121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd5d72-d040-4b9f-9f70-12151d205fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3f8febaf594522aa46977d0e59e342, user_id=8a89865b46b94140bde14022f2a547e2, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.152738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd5d72-d040-4b9f-9f70-12151d205fd3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.179708 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 971/1936] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:51 2022] PUT /identity/v3/projects/a33d01a937774e999dd56d6a5560640f/users/8757ef30a353447b9059719a2cf6d474/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:51.183484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.250133 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd96e2-5c15-45f8-a969-0012ed9bec79 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.254971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.255581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.255861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.256141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.256835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.258473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.259636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 966/1937] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:51 2022] PUT /identity/v3/projects/8a3f8febaf594522aa46977d0e59e342/users/8a89865b46b94140bde14022f2a547e2/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:51.268508 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 972/1938] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:51.270925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.271215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.271516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.273046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.284170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522c6d2-b9a0-4ffb-b0c3-65215bc8a58f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.300104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] MFA Rules not processed for user `8a89865b46b94140bde14022f2a547e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:51.301579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:51.323930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:51.341907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621c05f-51de-4efb-a7da-e28aa5166a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.342655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621c05f-51de-4efb-a7da-e28aa5166a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.343212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621c05f-51de-4efb-a7da-e28aa5166a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.343539 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621c05f-51de-4efb-a7da-e28aa5166a1a admin admin] PATH_INFO: `/v3/projects/a33d01a937774e999dd56d6a5560640f/users/8757ef30a353447b9059719a2cf6d474/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.345433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621c05f-51de-4efb-a7da-e28aa5166a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33d01a937774e999dd56d6a5560640f, user_id=8757ef30a353447b9059719a2cf6d474, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.349550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621c05f-51de-4efb-a7da-e28aa5166a1a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.360541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 973/1939] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:51 2022] PUT /identity/v3/projects/a33d01a937774e999dd56d6a5560640f/users/8757ef30a353447b9059719a2cf6d474/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:51.370687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.371153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.371395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.373459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 967/1940] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:39:51.374928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.386076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354889c-1f76-4c4a-9679-e46cfc99bd9b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.400912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] MFA Rules not processed for user `8757ef30a353447b9059719a2cf6d474`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:51.402202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:51.442345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.442993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.443353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.443557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.444196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.446138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.475637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 974/1941] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:51.487880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.490101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:51.501328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:51.521279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 968/1942] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:51 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:39:51.522407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.556199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88425c-0ba8-41c5-ab23-0283d5b6839d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.596025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513c487-8f6f-4359-bb2e-cda0278911cb admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.602110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.602859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.603468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.603666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.604429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.606604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.634840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.635645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.636134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.636287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.693876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 969/1943] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:51 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:51.700057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.700205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.701766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.713353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 975/1944] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:51.716199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2243b5-3888-4829-9bdd-ed1f4d95159a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.721101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.721867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.722152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.722980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.724322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.725998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.738895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 970/1945] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:51.745797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.756365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439fca64-00ba-464e-aaef-ec1ebaee9d79 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.760953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.761602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.761889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.762162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.762853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.764464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.775413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 976/1946] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:51.779586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.790806 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372f7ed-09df-4bfe-a7e5-936d2ee06571 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.796229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.796886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.797166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.797485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] PATH_INFO: `/v3/projects/ee2a96781ba942c393cc96056a336b44/users/5be243e91ceb4af69b8f5d3d0cc0ad20/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.812910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:51.813059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee2a96781ba942c393cc96056a336b44, user_id=5be243e91ceb4af69b8f5d3d0cc0ad20, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.814648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:51.815738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.827255 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71248ee-6ec0-4186-b044-87d7df978aac admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:51.830936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:51.831534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.832338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.832586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.834349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:51.891332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:51.892774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425a851-9356-4503-841c-459f709c3b6a service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:51.902378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 977/1947] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:39:51.916023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 971/1948] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:51 2022] PUT /identity/v3/projects/ee2a96781ba942c393cc96056a336b44/users/5be243e91ceb4af69b8f5d3d0cc0ad20/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:51.934741 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:51.935032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:51.935305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:51.942362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:51.964020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] MFA Rules not processed for user `5be243e91ceb4af69b8f5d3d0cc0ad20`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:51.965308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:52.033817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 978/1949] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:52.038971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:52.050506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:52.138959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899150c-34f1-416a-8fae-dff51dfcfc80 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:52.144323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:52.145531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.145823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.146092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.146773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:52.148918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:52.278851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 972/1950] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:39:52 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:52.283175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:52.305543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf685c2-2e89-4d28-bbbb-bafd72b20239 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:52.366217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:52.367235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.367524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.367804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.368493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:52.370564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:52.464700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 979/1951] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:39:52 2022] POST /identity/v3/users => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:39:52.469296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:52.480461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8b378-bfe7-4ed1-a6b7-3a5ea975169d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:52.485390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:52.486080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.486427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.487081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.487805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:52.490182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:52.505876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 973/1952] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:39:52.510369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:52.522337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf9ad3-fe3d-45d7-88ba-480e8f7aacc5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:52.529615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:52.530438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.530787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.531129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] PATH_INFO: `/v3/projects/ed565231ca6f4a379ee0e4766afa3010/users/933ac7ad54034d4d83ae1b7876b97ece/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.549979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed565231ca6f4a379ee0e4766afa3010, user_id=933ac7ad54034d4d83ae1b7876b97ece, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:52.553294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:52.623600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 980/1953] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:52 2022] PUT /identity/v3/projects/ed565231ca6f4a379ee0e4766afa3010/users/933ac7ad54034d4d83ae1b7876b97ece/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:39:52.624770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:52.635232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87608933-08da-4a78-95e1-812fead0aa6a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:52.705023 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd01de7-38e0-49d4-ba78-e461c8b2098b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:52.705688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd01de7-38e0-49d4-ba78-e461c8b2098b admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.708553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd01de7-38e0-49d4-ba78-e461c8b2098b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.708553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd01de7-38e0-49d4-ba78-e461c8b2098b admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.708553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd01de7-38e0-49d4-ba78-e461c8b2098b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:52.709037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd01de7-38e0-49d4-ba78-e461c8b2098b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:52.719832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 974/1954] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:39:52.724014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:52.735888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f05f9e-bfa5-46c7-b127-5d30db6c32bb admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:52.742165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:52.742893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.743121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.743397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] PATH_INFO: `/v3/projects/ed565231ca6f4a379ee0e4766afa3010/users/933ac7ad54034d4d83ae1b7876b97ece/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.747692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed565231ca6f4a379ee0e4766afa3010, user_id=933ac7ad54034d4d83ae1b7876b97ece, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:52.748341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:52.760583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 981/1955] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:39:52 2022] PUT /identity/v3/projects/ed565231ca6f4a379ee0e4766afa3010/users/933ac7ad54034d4d83ae1b7876b97ece/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:39:52.771095 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:52.771799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:52.777539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:52.802006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] MFA Rules not processed for user `933ac7ad54034d4d83ae1b7876b97ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:52.803254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:52.906369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:52.912865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 975/1956] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:39:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:52.970532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:53.027641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:39:53.037955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ae31-bee2-4da5-9f33-4a69d1d175a4 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:53.041645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:53.042258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:53.042549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:53.042815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:53.044583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:53.105958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:53.107412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be22f9-3318-4f74-b6e0-b8f12cb0ee9e service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:53.119691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 982/1957] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:39:53.210315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:53.221082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7c0c9-f30d-4085-a5bf-7fed3334d2bf None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:53.225822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863928f0-bb51-4461-add5-ca6f1c808e1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:53.226445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863928f0-bb51-4461-add5-ca6f1c808e1d service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:53.226730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863928f0-bb51-4461-add5-ca6f1c808e1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:53.227114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863928f0-bb51-4461-add5-ca6f1c808e1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:53.290931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863928f0-bb51-4461-add5-ca6f1c808e1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:53.292972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863928f0-bb51-4461-add5-ca6f1c808e1d service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:53.316808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 976/1958] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:39:54.715858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd780c-6a34-4239-9d4c-b7f723f05349 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:54.716710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd780c-6a34-4239-9d4c-b7f723f05349 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:54.717002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd780c-6a34-4239-9d4c-b7f723f05349 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:54.718567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 983/1959] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:39:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:39:54.738636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:54.739153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:54.739560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:54.763215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:39:54.764426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:54.873480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 977/1960] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:39:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:39:54.889965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd780c-6a34-4239-9d4c-b7f723f05349 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:39:54.891640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8bcd780c-6a34-4239-9d4c-b7f723f05349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:39:54.909679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd780c-6a34-4239-9d4c-b7f723f05349 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:39:54.915798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:39:54.916402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:39:54.916684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:39:54.916958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:39:54.926363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:39:54.927903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:39:54.941463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 984/1961] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:39:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:00.737242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:00.737242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588403c-0345-4abe-87d8-fa78d91f20c3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:00.740953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6284a-a23d-495c-a2e8-e361aa55c00e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:00.741620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6284a-a23d-495c-a2e8-e361aa55c00e service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:00.742107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6284a-a23d-495c-a2e8-e361aa55c00e service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:00.742389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6284a-a23d-495c-a2e8-e361aa55c00e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:00.801085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6284a-a23d-495c-a2e8-e361aa55c00e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:00.802581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6284a-a23d-495c-a2e8-e361aa55c00e service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:00.812248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 978/1962] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:02.222047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:02.318424 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0be0a-a47a-4af6-a246-bae25f68930a service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:02.325678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:02.325678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:02.325678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:02.325678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] PATH_INFO: `/v3/users/1ccdef4ac31b400c85b8b64c5a604e05` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:02.326993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ccdef4ac31b400c85b8b64c5a604e05)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:02.329250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:02.556825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ccdef4ac31b400c85b8b64c5a604e05'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:02.557509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ccdef4ac31b400c85b8b64c5a604e05'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:02.558129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ccdef4ac31b400c85b8b64c5a604e05'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:02.586524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ccdef4ac31b400c85b8b64c5a604e05'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:02.658019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ccdef4ac31b400c85b8b64c5a604e05'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:02.667171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:02.671697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 985/1963] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:02 2022] DELETE /identity/v3/users/1ccdef4ac31b400c85b8b64c5a604e05 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:02.676341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:02.676792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:02.676903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:02.697656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:02.699154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:02.719529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:02.772618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:02.773275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 979/1964] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:02.774695 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:02.783052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:02.787875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:02.833511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fbe0b-f3ab-492c-b52e-12d661a72b27 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:02.837602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:02.838245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:02.838552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:02.838808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:02.841738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:02.845715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657f1d8-b6b4-4e5f-ae72-2bf84ffb68c3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:02.850859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:02.851705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:02.852012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:02.852367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:02.865091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:02.866685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:02.877794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 980/1965] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:02.914258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:02.915719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:02.927762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 986/1966] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:03.397162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:03.398620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:03.416037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c3a9f-2ca4-4470-b732-28fff95a6b26 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:03.421882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:03.422219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:03.422298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:03.422527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] PATH_INFO: `/v3/projects/52d97e020205403898d03abb282ec539` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:03.424533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d97e020205403898d03abb282ec539)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:03.426288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:03.557470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d97e020205403898d03abb282ec539', 'request_id': 'req-a7aa9695-10c3-4c75-a874-5470b0634771'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.563562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d97e020205403898d03abb282ec539', 'request_id': 'req-a7aa9695-10c3-4c75-a874-5470b0634771'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.630637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d97e020205403898d03abb282ec539', 'request_id': 'req-a7aa9695-10c3-4c75-a874-5470b0634771'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.642417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] The token cache is being invalidate because project 52d97e020205403898d03abb282ec539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:03.642776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.643461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.645776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 981/1967] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:03 2022] DELETE /identity/v3/projects/52d97e020205403898d03abb282ec539 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:03.649868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:03.669499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d7013-cb02-4ffb-a3d3-36a5953fa6cb service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:03.737755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:03.739030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:03.739313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:03.739587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] PATH_INFO: `/v3/users/d0121578fc3a42ba8213556293fd2b95` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:03.740773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0121578fc3a42ba8213556293fd2b95)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:03.742896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:03.899591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0121578fc3a42ba8213556293fd2b95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.900281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0121578fc3a42ba8213556293fd2b95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.900890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0121578fc3a42ba8213556293fd2b95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.905512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:03.913717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0121578fc3a42ba8213556293fd2b95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.971988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0121578fc3a42ba8213556293fd2b95'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:03.984859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 987/1968] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:03 2022] DELETE /identity/v3/users/d0121578fc3a42ba8213556293fd2b95 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:04.397458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:04.399545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:04.414931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa9695-10c3-4c75-a874-5470b0634771 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:04.467287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:04.476678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:04.477332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:04.477612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:04.477881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] PATH_INFO: `/v3/projects/55cb77d13eaf4a358768312054d223a5` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:04.479141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cb77d13eaf4a358768312054d223a5)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:04.481055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:04.631218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cb77d13eaf4a358768312054d223a5', 'request_id': 'req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:04.638387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cb77d13eaf4a358768312054d223a5', 'request_id': 'req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:04.734041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cb77d13eaf4a358768312054d223a5', 'request_id': 'req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:04.750812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] The token cache is being invalidate because project 55cb77d13eaf4a358768312054d223a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:04.751273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:04.752272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:04.755299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 982/1969] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:04 2022] DELETE /identity/v3/projects/55cb77d13eaf4a358768312054d223a5 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:04.759300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:04.761475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:04.785324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b64439-e60d-47c9-a2c2-e1405eb3b31c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:04.880799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:04.881619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:04.881980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:04.882336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] PATH_INFO: `/v3/users/8757ef30a353447b9059719a2cf6d474` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:04.883590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8757ef30a353447b9059719a2cf6d474)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:04.885732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:05.091177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8757ef30a353447b9059719a2cf6d474'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.091823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8757ef30a353447b9059719a2cf6d474'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.092433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8757ef30a353447b9059719a2cf6d474'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.105573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8757ef30a353447b9059719a2cf6d474'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.200857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-08c2e518-6e96-4539-be6f-5b7ad42b6be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8757ef30a353447b9059719a2cf6d474'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.214137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 988/1970] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:04 2022] DELETE /identity/v3/users/8757ef30a353447b9059719a2cf6d474 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:05.625331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:05.626840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:05.641424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48cf6a-a5b3-4e31-a5c6-008bc4da2a3e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:05.699065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:05.699719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:05.700007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:05.700282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] PATH_INFO: `/v3/projects/a33d01a937774e999dd56d6a5560640f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:05.701473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33d01a937774e999dd56d6a5560640f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:05.703493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:05.819007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33d01a937774e999dd56d6a5560640f', 'request_id': 'req-2064f2b4-aceb-46ac-b278-cc8c58458b2f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.825486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33d01a937774e999dd56d6a5560640f', 'request_id': 'req-2064f2b4-aceb-46ac-b278-cc8c58458b2f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.892327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33d01a937774e999dd56d6a5560640f', 'request_id': 'req-2064f2b4-aceb-46ac-b278-cc8c58458b2f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.908255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] The token cache is being invalidate because project a33d01a937774e999dd56d6a5560640f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:05.908656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.909463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2064f2b4-aceb-46ac-b278-cc8c58458b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:05.911836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 983/1971] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:05 2022] DELETE /identity/v3/projects/a33d01a937774e999dd56d6a5560640f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:05.917194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:05.917718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:05.917797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:05.938481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:05.939879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:05.992143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:06.008640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 989/1972] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:06.016946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.017249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.017529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.038969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:06.040324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:06.048674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 984/1973] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:06.053185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.063129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f0396-003d-40b3-9ac8-0f85b4cd0568 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.068305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.068921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.069232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.069504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.070204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:06.070505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.072113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.090503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 990/1974] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:40:06.092797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.118966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd1e15-b0ca-4cbd-b470-3a24850a9496 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.125683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.126426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.126777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.127120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.127872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.130818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.136189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:06.237670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 985/1975] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:06 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:06.242297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.252348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f82bfd-8eea-4c5d-9165-06bd92d8c8f7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.310374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.310915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.311197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.311474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.312142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.314158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.424762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 991/1976] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:06 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:06.429224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.439716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398693b-adb4-472e-a47d-8cde205d78ac admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.445270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.445919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.446194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.446472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.447171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.448776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.459367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 986/1977] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:06.464649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.474961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94994592-9117-4822-b464-7607daac3c36 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.479583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6418450-dca5-4b2d-9dd0-2a5ed51cbbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.480199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418450-dca5-4b2d-9dd0-2a5ed51cbbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.480469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418450-dca5-4b2d-9dd0-2a5ed51cbbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.480743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418450-dca5-4b2d-9dd0-2a5ed51cbbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.481469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418450-dca5-4b2d-9dd0-2a5ed51cbbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.483071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418450-dca5-4b2d-9dd0-2a5ed51cbbe3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.493962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 992/1978] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:06.498021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.508303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682feb-16d6-473c-a5b7-0380842a0873 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.512824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.513509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.513788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.514060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] PATH_INFO: `/v3/projects/509836426eac467789fb32741dde7458/users/c451e6aa6eee4be1b9b07f0a37e38468/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.529337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] RBAC: Authorizing `identity:create_grant(project_id=509836426eac467789fb32741dde7458, user_id=c451e6aa6eee4be1b9b07f0a37e38468, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.532010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.601405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 987/1979] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:06 2022] PUT /identity/v3/projects/509836426eac467789fb32741dde7458/users/c451e6aa6eee4be1b9b07f0a37e38468/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:06.612874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.616499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.616499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.641256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] MFA Rules not processed for user `c451e6aa6eee4be1b9b07f0a37e38468`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:06.642474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:06.707671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 993/1980] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:06.719345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.721158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:06.779825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731dde1-bd62-4b24-9165-4cf8cc73510a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.783372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c48ec8-d809-4f9e-ba13-b41da1115331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.783983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48ec8-d809-4f9e-ba13-b41da1115331 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.784259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48ec8-d809-4f9e-ba13-b41da1115331 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.784523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c48ec8-d809-4f9e-ba13-b41da1115331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.794118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48ec8-d809-4f9e-ba13-b41da1115331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.795591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c48ec8-d809-4f9e-ba13-b41da1115331 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:06.804227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 988/1981] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:06.825179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:06.878079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:06.896904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9275607-a62f-43e6-94ab-a284d5087406 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:06.901857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:06.902509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:06.902798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:06.903330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] PATH_INFO: `/v3/users/c451e6aa6eee4be1b9b07f0a37e38468` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:06.904530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c451e6aa6eee4be1b9b07f0a37e38468)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:06.906591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:07.111354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c451e6aa6eee4be1b9b07f0a37e38468'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:07.112324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c451e6aa6eee4be1b9b07f0a37e38468'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:07.113655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c451e6aa6eee4be1b9b07f0a37e38468'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:07.126308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c451e6aa6eee4be1b9b07f0a37e38468'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:07.195402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c451e6aa6eee4be1b9b07f0a37e38468'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:07.212849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 994/1982] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:06 2022] DELETE /identity/v3/users/c451e6aa6eee4be1b9b07f0a37e38468 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:07.217438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:07.217892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:07.218311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:07.237836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:07.239530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:07.312220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 989/1983] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:07.323329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:07.324816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:07.397105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd23ef8-87d7-44c7-b8ef-7574b69afc5b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:07.401801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250ddf2-7d07-4966-91dc-43cc82c7a106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:07.402495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250ddf2-7d07-4966-91dc-43cc82c7a106 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:07.402841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250ddf2-7d07-4966-91dc-43cc82c7a106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:07.403193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250ddf2-7d07-4966-91dc-43cc82c7a106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:07.413942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250ddf2-7d07-4966-91dc-43cc82c7a106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:07.415413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250ddf2-7d07-4966-91dc-43cc82c7a106 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:07.426333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 995/1984] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:07.819944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:07.821469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:07.834190 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472b223-dfa7-4e70-b3ef-a30bcf5ccfbf None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:07.839104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:07.839930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:07.840209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:07.845476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] PATH_INFO: `/v3/projects/509836426eac467789fb32741dde7458` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:07.845476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509836426eac467789fb32741dde7458)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:07.845476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:07.977759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:07.978972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509836426eac467789fb32741dde7458', 'request_id': 'req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:07.985065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509836426eac467789fb32741dde7458', 'request_id': 'req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:08.068944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509836426eac467789fb32741dde7458', 'request_id': 'req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:08.080055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] The token cache is being invalidate because project 509836426eac467789fb32741dde7458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:08.080343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:08.080971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-40e12dd7-5cde-420b-bd5c-d2f2fda77d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:08.083212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 990/1985] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:07 2022] DELETE /identity/v3/projects/509836426eac467789fb32741dde7458 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:08.088008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.088323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.088631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.109127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:08.110378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:08.169173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 996/1986] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:08.177030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.177357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.177749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.197755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:08.198955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:08.207639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 991/1987] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:08.211591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.221216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592150-f5b5-487c-8638-a172a29bc3f7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.225883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.226524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.226806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.227080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.227773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:08.228074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.229721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:08.231194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:08.242537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 997/1988] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:40:08.247642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.257821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f3303-4c4b-4565-8b83-fdf0d3f56c2a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.262629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.263240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.263687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.263794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.264481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.266458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:08.372149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 992/1989] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:08 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:08.376062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.389149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16790f50-bef9-4187-976e-597516b00ea9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.462292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.462927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.463214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.463494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.464164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.466169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:08.573696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 998/1990] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:08 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:08.577743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.588358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce2a9-dc01-452d-b200-b6a5d904667d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.593509 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.594290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.597871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.597871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.597871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.597871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:08.608543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 993/1991] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:08.613410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.626304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df683da-d41a-4e81-bb3b-5ff72598d7ca admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.632405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94a135-5465-4937-9e9c-98b0ad2b7a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.633170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94a135-5465-4937-9e9c-98b0ad2b7a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.633512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94a135-5465-4937-9e9c-98b0ad2b7a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.633879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94a135-5465-4937-9e9c-98b0ad2b7a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.634590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94a135-5465-4937-9e9c-98b0ad2b7a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.636526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da94a135-5465-4937-9e9c-98b0ad2b7a42 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:08.656668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 999/1992] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:08.660824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.670818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e398-c410-4821-ad9a-1bf3049ec888 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.678837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.678837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.678837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.678837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] PATH_INFO: `/v3/projects/caf8f1107b1c401dbdaaa2e98e26d0a4/users/15879ae6a1b148d4966d34db663756d2/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.694021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf8f1107b1c401dbdaaa2e98e26d0a4, user_id=15879ae6a1b148d4966d34db663756d2, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.696804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:08.784476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 994/1993] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:08 2022] PUT /identity/v3/projects/caf8f1107b1c401dbdaaa2e98e26d0a4/users/15879ae6a1b148d4966d34db663756d2/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:08.796236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.796523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.796813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.826218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] MFA Rules not processed for user `15879ae6a1b148d4966d34db663756d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:08.827413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:08.891751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1000/1994] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:08.896297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:08.959362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a4c8a-9dc1-476e-b24c-2f0ca0df5f8a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:08.964227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:08.964845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:08.965130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:08.965438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:08.966116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:08.968037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.047875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 995/1995] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:08 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:09.052147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:09.062948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e584ea6-196f-4edf-840a-f57aecb69782 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:09.119008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:09.119623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.119905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.120178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.120834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:09.122789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.131329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:09.201240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1001/1996] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:09 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:09.205098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:09.216592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603f22-5276-4389-a6ae-19f0b3ca2e51 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:09.221273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:09.221885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.222895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.223267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.223966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:09.225755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.236358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 996/1997] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:09.240803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:09.250736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3a4fe-b2e5-4766-b7ae-c5f5d3d5bae7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:09.255419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:09.256074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.256360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.256641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] PATH_INFO: `/v3/projects/8c912b524a36424ca09d7c38942c06c1/users/4d61c7cf9a0f49a19d144f5e70ddd4c2/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.271803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c912b524a36424ca09d7c38942c06c1, user_id=4d61c7cf9a0f49a19d144f5e70ddd4c2, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:09.274538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.379014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1002/1998] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:09 2022] PUT /identity/v3/projects/8c912b524a36424ca09d7c38942c06c1/users/4d61c7cf9a0f49a19d144f5e70ddd4c2/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:09.383742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:09.400267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8dfaa-b286-4f72-9891-d213937d2236 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:09.458122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:09.470679 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78bf6-569d-4d14-a4e2-7b7c1580428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:09.471313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78bf6-569d-4d14-a4e2-7b7c1580428f admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.471591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78bf6-569d-4d14-a4e2-7b7c1580428f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.471867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78bf6-569d-4d14-a4e2-7b7c1580428f admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.472566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78bf6-569d-4d14-a4e2-7b7c1580428f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:09.474454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78bf6-569d-4d14-a4e2-7b7c1580428f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.485760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 997/1999] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:09.490197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:09.501787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5bf4-75cf-45ae-8560-a7ef0b69b0bd admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:09.507560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:09.508442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.508718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.509130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] PATH_INFO: `/v3/projects/8c912b524a36424ca09d7c38942c06c1/users/4d61c7cf9a0f49a19d144f5e70ddd4c2/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.518222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c912b524a36424ca09d7c38942c06c1, user_id=4d61c7cf9a0f49a19d144f5e70ddd4c2, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:09.518222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.528726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1003/2000] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:09 2022] PUT /identity/v3/projects/8c912b524a36424ca09d7c38942c06c1/users/4d61c7cf9a0f49a19d144f5e70ddd4c2/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:09.546273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.546556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.546876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.575325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] MFA Rules not processed for user `4d61c7cf9a0f49a19d144f5e70ddd4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:09.576539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:09.647347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 998/2001] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:09.660315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:09.662616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:09.741998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07683cf6-9900-427a-bf03-c038e19139f0 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:09.745791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:09.746472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:09.746754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:09.747056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:09.815000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:09.817932 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:09.829499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1004/2002] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:19.680971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=42 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:19.824177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:20.566759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:20.630853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06b94e-67cb-4a62-9b65-845761ea25ad None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:20.635633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:20.636304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:20.636586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:20.636875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] PATH_INFO: `/v3/users/15879ae6a1b148d4966d34db663756d2` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:20.638166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15879ae6a1b148d4966d34db663756d2)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:20.640126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:20.822425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:20.826652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:20.857419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ae00-fc62-4561-a5b4-37d2a1a30f20 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:20.863030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:20.863803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:20.864155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:20.864672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] PATH_INFO: `/v3/users/8a89865b46b94140bde14022f2a547e2` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:20.866222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a89865b46b94140bde14022f2a547e2)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:20.869309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:20.902117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15879ae6a1b148d4966d34db663756d2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:20.916958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15879ae6a1b148d4966d34db663756d2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:20.917622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15879ae6a1b148d4966d34db663756d2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:20.948779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15879ae6a1b148d4966d34db663756d2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.019449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a6cd48a9-383a-4033-98ba-ef53daddc91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15879ae6a1b148d4966d34db663756d2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.032135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 999/2003] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:20 2022] DELETE /identity/v3/users/15879ae6a1b148d4966d34db663756d2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:21.035085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:21.035375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:21.035649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:21.075253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:21.080597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:21.137328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a89865b46b94140bde14022f2a547e2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.138918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a89865b46b94140bde14022f2a547e2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.140346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a89865b46b94140bde14022f2a547e2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.164354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a89865b46b94140bde14022f2a547e2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.196300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1000/2004] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:21.210357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:21.223016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:21.248661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5b45751b-27a6-471c-a3e7-dfcd7974ec87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a89865b46b94140bde14022f2a547e2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:21.264758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:21.266554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1005/2005] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:20 2022] DELETE /identity/v3/users/8a89865b46b94140bde14022f2a547e2 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:21.271889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:21.272184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:21.272464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:21.294344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:21.295574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:21.307435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996aaca-d2bb-44d4-9ab1-6f3fa16b74cf None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:21.373334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:21.374138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:21.374514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:21.374939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:21.383743 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1006/2006] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:21.393292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:21.399479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:21.414088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ce604-de96-4e7a-8873-76d3d7cb4954 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:21.419382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:21.420284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:21.420647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:21.421017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:21.437745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:21.438043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:21.475582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:21.477147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:21.479021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1007/2007] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:21.489628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1001/2008] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:22.088916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:22.090488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:22.104603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a72a0f-ae02-423f-9d39-c414b3150caa service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:22.110341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:22.111909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:22.112259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:22.112802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] PATH_INFO: `/v3/projects/caf8f1107b1c401dbdaaa2e98e26d0a4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:22.115162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf8f1107b1c401dbdaaa2e98e26d0a4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:22.117090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:22.217340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:22.219114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:22.237179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61aff3a-501b-467f-b716-6c5962cc7b42 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:22.243077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:22.243737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:22.244019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:22.244391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] PATH_INFO: `/v3/projects/8a3f8febaf594522aa46977d0e59e342` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:22.245630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3f8febaf594522aa46977d0e59e342)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:22.247759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:22.259384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf8f1107b1c401dbdaaa2e98e26d0a4', 'request_id': 'req-4763aa40-d504-4540-a1e8-15d91640f2aa'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.278559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf8f1107b1c401dbdaaa2e98e26d0a4', 'request_id': 'req-4763aa40-d504-4540-a1e8-15d91640f2aa'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.280510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:22.361650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf8f1107b1c401dbdaaa2e98e26d0a4', 'request_id': 'req-4763aa40-d504-4540-a1e8-15d91640f2aa'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.383850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] The token cache is being invalidate because project caf8f1107b1c401dbdaaa2e98e26d0a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:22.384349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.385418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.389772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1008/2009] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:22 2022] DELETE /identity/v3/projects/caf8f1107b1c401dbdaaa2e98e26d0a4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:22.399685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:22.399685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:22.400060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3f8febaf594522aa46977d0e59e342', 'request_id': 'req-126d5e0b-9706-4e7e-b983-1ca3f790f44b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.408474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3f8febaf594522aa46977d0e59e342', 'request_id': 'req-126d5e0b-9706-4e7e-b983-1ca3f790f44b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.480844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763aa40-d504-4540-a1e8-15d91640f2aa admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:22.489258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3f8febaf594522aa46977d0e59e342', 'request_id': 'req-126d5e0b-9706-4e7e-b983-1ca3f790f44b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.506505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] The token cache is being invalidate because project 8a3f8febaf594522aa46977d0e59e342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:22.506808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.507669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.512773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1002/2010] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:22 2022] DELETE /identity/v3/projects/8a3f8febaf594522aa46977d0e59e342 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:22.513852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:22.515519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:22.531184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:22.556541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:22.557330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:22.557618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:22.558343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] PATH_INFO: `/v3/users/4d61c7cf9a0f49a19d144f5e70ddd4c2` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:22.559312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d61c7cf9a0f49a19d144f5e70ddd4c2)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:22.561344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:22.598404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d5e0b-9706-4e7e-b983-1ca3f790f44b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:22.602978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:22.610350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:22.610350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:22.610350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] PATH_INFO: `/v3/users/5be243e91ceb4af69b8f5d3d0cc0ad20` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:22.610350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be243e91ceb4af69b8f5d3d0cc0ad20)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:22.610350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:22.822435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d61c7cf9a0f49a19d144f5e70ddd4c2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.823077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d61c7cf9a0f49a19d144f5e70ddd4c2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.823600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d61c7cf9a0f49a19d144f5e70ddd4c2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.836436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d61c7cf9a0f49a19d144f5e70ddd4c2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.875081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be243e91ceb4af69b8f5d3d0cc0ad20'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.883377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be243e91ceb4af69b8f5d3d0cc0ad20'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.884078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be243e91ceb4af69b8f5d3d0cc0ad20'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.901416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be243e91ceb4af69b8f5d3d0cc0ad20'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.920709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d61c7cf9a0f49a19d144f5e70ddd4c2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.934063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1009/2011] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:22 2022] DELETE /identity/v3/users/4d61c7cf9a0f49a19d144f5e70ddd4c2 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:22.972289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be243e91ceb4af69b8f5d3d0cc0ad20'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:22.978912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1003/2012] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:22 2022] DELETE /identity/v3/users/5be243e91ceb4af69b8f5d3d0cc0ad20 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:23.399633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:23.401241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:23.412307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:23.415160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:23.418546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:23.472247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf58f0-f3f6-4d79-b33b-264de810c683 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:23.496159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be3eee-c084-4e56-ad75-4270896a1d44 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:23.501438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:23.502108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.502523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.502799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] PATH_INFO: `/v3/projects/ee2a96781ba942c393cc96056a336b44` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:23.504355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee2a96781ba942c393cc96056a336b44)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:23.506359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:23.532406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:23.533093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.533420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.533699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] PATH_INFO: `/v3/projects/8c912b524a36424ca09d7c38942c06c1` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:23.535536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c912b524a36424ca09d7c38942c06c1)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:23.537518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:23.599613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:23.611885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee2a96781ba942c393cc96056a336b44', 'request_id': 'req-2eb468e2-9474-4471-adc2-918c425410db'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.618110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee2a96781ba942c393cc96056a336b44', 'request_id': 'req-2eb468e2-9474-4471-adc2-918c425410db'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.641109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c912b524a36424ca09d7c38942c06c1', 'request_id': 'req-070f886d-c31b-4551-8354-c7094e61bbb2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.649321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c912b524a36424ca09d7c38942c06c1', 'request_id': 'req-070f886d-c31b-4551-8354-c7094e61bbb2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.675185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee2a96781ba942c393cc96056a336b44', 'request_id': 'req-2eb468e2-9474-4471-adc2-918c425410db'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.686406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] The token cache is being invalidate because project ee2a96781ba942c393cc96056a336b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:23.686695 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.687325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.688871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1004/2013] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:23 2022] DELETE /identity/v3/projects/ee2a96781ba942c393cc96056a336b44 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:23.693297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:23.694670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:23.749736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c912b524a36424ca09d7c38942c06c1', 'request_id': 'req-070f886d-c31b-4551-8354-c7094e61bbb2'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.761013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb468e2-9474-4471-adc2-918c425410db admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:23.762914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] The token cache is being invalidate because project 8c912b524a36424ca09d7c38942c06c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:23.763205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.763895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-070f886d-c31b-4551-8354-c7094e61bbb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:23.769855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1010/2014] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:23 2022] DELETE /identity/v3/projects/8c912b524a36424ca09d7c38942c06c1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:23.771921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f83a-bc65-4d55-99a4-e7d0c522001a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.772234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f83a-bc65-4d55-99a4-e7d0c522001a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.772515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8f83a-bc65-4d55-99a4-e7d0c522001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:23.797103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-02a8f83a-bc65-4d55-99a4-e7d0c522001a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:23.798528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-02a8f83a-bc65-4d55-99a4-e7d0c522001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:23.837052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:23.837052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.837052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.837052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] PATH_INFO: `/v3/users/933ac7ad54034d4d83ae1b7876b97ece` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:23.841022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933ac7ad54034d4d83ae1b7876b97ece)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:23.843240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:23.869189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:23.886943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1011/2015] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:23.895221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.895513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.895790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:23.915763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:23.917663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:23.926845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1012/2016] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:23.930920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:23.939642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524490d5-fb8c-4a94-8689-f36b1b37424a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:23.944588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:23.945226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.947754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.947754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:23.947754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:23.947754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:23.948322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:23.965503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1013/2017] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:40:23.967173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:23.981346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aadabf-8ff7-43e5-88bd-8f2d7f059bc8 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:23.999604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:23.999604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:23.999604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:23.999604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.000973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.000973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.033570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933ac7ad54034d4d83ae1b7876b97ece'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.041368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933ac7ad54034d4d83ae1b7876b97ece'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.042105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933ac7ad54034d4d83ae1b7876b97ece'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.050104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:24.054720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933ac7ad54034d4d83ae1b7876b97ece'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.082814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1014/2018] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:23 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:24.086824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:24.088441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:24.143438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933ac7ad54034d4d83ae1b7876b97ece'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.149901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1005/2019] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:23 2022] DELETE /identity/v3/users/933ac7ad54034d4d83ae1b7876b97ece => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:24.153802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c417efe-649e-41ba-9858-f197d1da7aba admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:24.214225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:24.214882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.215166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.215441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.216110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.218202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.287223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1015/2020] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:24 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:40:24.291327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:24.302303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29f36d-5484-4dd3-a51e-193dab66e574 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:24.307030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:24.310374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.310374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.310374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.310374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.310704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.321699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1006/2021] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:24.330368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:24.340789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12006f-fc6a-4f87-b509-4cf49341d92b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:24.346410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27b8fe-5549-4198-9b18-7b4d68411f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:24.347701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27b8fe-5549-4198-9b18-7b4d68411f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.348144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27b8fe-5549-4198-9b18-7b4d68411f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.348422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27b8fe-5549-4198-9b18-7b4d68411f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.349126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27b8fe-5549-4198-9b18-7b4d68411f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.351173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27b8fe-5549-4198-9b18-7b4d68411f7a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.363092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1016/2022] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:24.367749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:24.377770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3011be-ff9f-459f-9fff-4947e0becd70 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:24.384953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:24.386141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.386837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.387663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] PATH_INFO: `/v3/projects/1c4daab5960f4c98874faea2066b1909/users/d8a3eed3ee0b42f69948aced53729d6f/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.410217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4daab5960f4c98874faea2066b1909, user_id=d8a3eed3ee0b42f69948aced53729d6f, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.413210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.488073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1007/2023] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:24 2022] PUT /identity/v3/projects/1c4daab5960f4c98874faea2066b1909/users/d8a3eed3ee0b42f69948aced53729d6f/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:24.493321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.493712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.493875 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.513450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:24.518256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:24.541640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:24.574267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:24.579602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1017/2024] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:24.580268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:24.587595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:24.588871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:24.599096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fc04a-07ca-4c39-86a5-0194b670150b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:24.604702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:24.605443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.605729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.606004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] PATH_INFO: `/v3/projects/ed565231ca6f4a379ee0e4766afa3010` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.607176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed565231ca6f4a379ee0e4766afa3010)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.609165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.652178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604198f0-3b73-4821-b68b-1d4f508dc72e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:24.655802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b80b21-8e2c-4393-9f0d-b7ab99c35e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:24.656503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b80b21-8e2c-4393-9f0d-b7ab99c35e6c service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.656790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b80b21-8e2c-4393-9f0d-b7ab99c35e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.657067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b80b21-8e2c-4393-9f0d-b7ab99c35e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.667484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b80b21-8e2c-4393-9f0d-b7ab99c35e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:24.668927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b80b21-8e2c-4393-9f0d-b7ab99c35e6c service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:24.680479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1018/2025] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:24.715987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed565231ca6f4a379ee0e4766afa3010', 'request_id': 'req-c21e774a-62f7-41c7-b91e-18a746c501cc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.722767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed565231ca6f4a379ee0e4766afa3010', 'request_id': 'req-c21e774a-62f7-41c7-b91e-18a746c501cc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.812765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed565231ca6f4a379ee0e4766afa3010', 'request_id': 'req-c21e774a-62f7-41c7-b91e-18a746c501cc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.823957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] The token cache is being invalidate because project ed565231ca6f4a379ee0e4766afa3010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:24.824243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.824899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-c21e774a-62f7-41c7-b91e-18a746c501cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:24.836503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1008/2026] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:24 2022] DELETE /identity/v3/projects/ed565231ca6f4a379ee0e4766afa3010 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:24.842020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.842345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.842584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.877084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:24.879175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:24.943470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1019/2027] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:24.952414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:24.952753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:24.953134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:24.975742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:24.977088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:24.985370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1009/2028] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:24.990059 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.003713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52f946-04d9-40ac-af6d-2fd264e0baf4 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.008996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.009667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.009949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.010381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.011114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:25.011482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.013040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.028321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1020/2029] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:40:25.032475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.043233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602027ca-94fa-49a1-89b8-f6ff2df7387f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.048392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.049019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.049388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.049664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.050914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.052959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.137232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1010/2030] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:25 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:25.138945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.149088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd940a-7e93-4698-8839-7d450af39f5b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.215263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.215804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.216105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.216383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.217049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.219025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.228749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:25.356324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1021/2031] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:25 2022] POST /identity/v3/users => generated 292 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:25.360331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.371720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e3824-3247-4227-b48e-08ec5b7a52c3 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.376663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.381430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.381430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.381430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.381430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.381430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.392279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1011/2032] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:25.397011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.407274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446e49b-5f9f-44de-b5e4-99b808d22b73 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.412651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715abb10-7171-4e3c-a5f2-81bc053e3679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.413298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715abb10-7171-4e3c-a5f2-81bc053e3679 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.413584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715abb10-7171-4e3c-a5f2-81bc053e3679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.413860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715abb10-7171-4e3c-a5f2-81bc053e3679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.414559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715abb10-7171-4e3c-a5f2-81bc053e3679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.416200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715abb10-7171-4e3c-a5f2-81bc053e3679 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.427345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1022/2033] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:25.431439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.442835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b7518-e09d-412f-be79-4e3436d2219d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.447971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.449252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.449537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.449818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] PATH_INFO: `/v3/projects/9d5e4f28f5104cd7848170f924bf04c6/users/5864cfadc77f40e7b510d181f5f3cd6d/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.467344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d5e4f28f5104cd7848170f924bf04c6, user_id=5864cfadc77f40e7b510d181f5f3cd6d, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.469526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.538931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1012/2034] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:25 2022] PUT /identity/v3/projects/9d5e4f28f5104cd7848170f924bf04c6/users/5864cfadc77f40e7b510d181f5f3cd6d/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:25.549192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b05eb5-2c02-4949-9b3c-112744868aca None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.549653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b05eb5-2c02-4949-9b3c-112744868aca None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.549787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b05eb5-2c02-4949-9b3c-112744868aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.578369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b4b05eb5-2c02-4949-9b3c-112744868aca None None] MFA Rules not processed for user `5864cfadc77f40e7b510d181f5f3cd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:25.579587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b4b05eb5-2c02-4949-9b3c-112744868aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:25.646799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1023/2035] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:25.658214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:25.660635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:25.692901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:25.720129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35415dd5-9d65-4256-946c-7b7bf1950c06 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:25.724544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:25.725172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:25.725485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:25.725759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:25.734383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:25.736029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:25.747332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1013/2036] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:26.001837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008de00-cf5f-4151-9441-f3e2160eadad None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:26.002128 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008de00-cf5f-4151-9441-f3e2160eadad None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:26.002406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008de00-cf5f-4151-9441-f3e2160eadad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:26.034215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8008de00-cf5f-4151-9441-f3e2160eadad None None] MFA Rules not processed for user `d8a3eed3ee0b42f69948aced53729d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:26.036111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8008de00-cf5f-4151-9441-f3e2160eadad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:26.110093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1024/2037] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:26.126246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:26.127613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:26.146456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072f0be-0fa4-4e01-9913-938b62718a8a service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:26.146456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:26.146456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:26.146758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:26.147037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:26.155735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:26.157176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:26.194227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1014/2038] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:36.107103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:INSERT=2,keystone:SELECT=71 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:36.162710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=19 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:40.558723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f191-40e1-4d05-b858-4ee0d7f4b43e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:40.559236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f191-40e1-4d05-b858-4ee0d7f4b43e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:40.559805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19f191-40e1-4d05-b858-4ee0d7f4b43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:40.579191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bf19f191-40e1-4d05-b858-4ee0d7f4b43e None None] MFA Rules not processed for user `d8a3eed3ee0b42f69948aced53729d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:40.580422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bf19f191-40e1-4d05-b858-4ee0d7f4b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:40.589717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1025/2039] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:41.494022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:41.602162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776daa7-58c6-4b51-bfbf-8f883893f0dc service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:41.607840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:41.608497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:41.608782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:41.609052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] PATH_INFO: `/v3/users/5864cfadc77f40e7b510d181f5f3cd6d` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:41.610474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5864cfadc77f40e7b510d181f5f3cd6d)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:41.614524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:41.812368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5864cfadc77f40e7b510d181f5f3cd6d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:41.821771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5864cfadc77f40e7b510d181f5f3cd6d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:41.822652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5864cfadc77f40e7b510d181f5f3cd6d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:41.842528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5864cfadc77f40e7b510d181f5f3cd6d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:41.932506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5864cfadc77f40e7b510d181f5f3cd6d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:41.940701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1015/2040] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:41 2022] DELETE /identity/v3/users/5864cfadc77f40e7b510d181f5f3cd6d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:41.945649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286fe5a-6ac8-4592-a250-e40b99831426 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:41.946210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286fe5a-6ac8-4592-a250-e40b99831426 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:41.946625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286fe5a-6ac8-4592-a250-e40b99831426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:41.990003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1286fe5a-6ac8-4592-a250-e40b99831426 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:42.000533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1286fe5a-6ac8-4592-a250-e40b99831426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:42.099651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1026/2041] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:42.110279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:42.116310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:42.201982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877404a3-0403-4c26-be6a-ec67da5dc1b7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:42.207847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98e9a2-d4f6-49b7-9e8a-f8a3f1a45960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:42.208506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98e9a2-d4f6-49b7-9e8a-f8a3f1a45960 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:42.208810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98e9a2-d4f6-49b7-9e8a-f8a3f1a45960 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:42.209127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98e9a2-d4f6-49b7-9e8a-f8a3f1a45960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:42.226293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98e9a2-d4f6-49b7-9e8a-f8a3f1a45960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:42.226293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98e9a2-d4f6-49b7-9e8a-f8a3f1a45960 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:42.236676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1016/2042] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:42.516666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204a3c8-ba36-4d00-8928-ef2b3d58d91f None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:42.517694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204a3c8-ba36-4d00-8928-ef2b3d58d91f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:42.520999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204a3c8-ba36-4d00-8928-ef2b3d58d91f None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:42.520999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1027/2043] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:40:42.760692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:42.761249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:42.761372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:42.781665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:42.782899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:42.802232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:42.828729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204a3c8-ba36-4d00-8928-ef2b3d58d91f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:42.831321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a204a3c8-ba36-4d00-8928-ef2b3d58d91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:42.863649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1017/2044] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:40:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:42.874413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:42.876004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:42.878519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204a3c8-ba36-4d00-8928-ef2b3d58d91f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:42.886243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:42.890988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:42.891387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:42.891599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] PATH_INFO: `/v3/projects/9d5e4f28f5104cd7848170f924bf04c6` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:42.892918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d5e4f28f5104cd7848170f924bf04c6)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:42.893046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be527e42-3f22-4f3d-ac7b-4302f6faa194 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:42.895818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:42.904398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bcaa77-f951-472f-8c68-d843ef796522 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:42.905455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcaa77-f951-472f-8c68-d843ef796522 service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:42.905885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcaa77-f951-472f-8c68-d843ef796522 service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:42.908821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bcaa77-f951-472f-8c68-d843ef796522 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:42.920103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcaa77-f951-472f-8c68-d843ef796522 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:42.921330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bcaa77-f951-472f-8c68-d843ef796522 service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:42.937017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1018/2045] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:43.066660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d5e4f28f5104cd7848170f924bf04c6', 'request_id': 'req-79b90145-8bf4-4d09-8065-285e3a40efd3'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:43.074819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d5e4f28f5104cd7848170f924bf04c6', 'request_id': 'req-79b90145-8bf4-4d09-8065-285e3a40efd3'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:43.148827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d5e4f28f5104cd7848170f924bf04c6', 'request_id': 'req-79b90145-8bf4-4d09-8065-285e3a40efd3'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:43.161785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] The token cache is being invalidate because project 9d5e4f28f5104cd7848170f924bf04c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:43.162197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:43.163026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-79b90145-8bf4-4d09-8065-285e3a40efd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:43.165226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1028/2046] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:42 2022] DELETE /identity/v3/projects/9d5e4f28f5104cd7848170f924bf04c6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:43.170381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60514bb-696a-4448-986a-5550282390d5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.170671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60514bb-696a-4448-986a-5550282390d5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.171025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60514bb-696a-4448-986a-5550282390d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.191906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f60514bb-696a-4448-986a-5550282390d5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:43.193035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f60514bb-696a-4448-986a-5550282390d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:43.257468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1019/2047] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:43.262096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.262388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.262665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.286266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:43.287473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:43.295945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1029/2048] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:43.300583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60514bb-696a-4448-986a-5550282390d5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.310540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60514bb-696a-4448-986a-5550282390d5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.315176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.315917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.316199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.316475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.317910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:43.317910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.319125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.331354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1020/2049] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:40:43.335654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.345055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7d13a-1317-4bbc-8e11-351d7ca2aa97 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.350472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.351088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.352464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.352464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.352464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.354216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.432648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1030/2050] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:43 2022] POST /identity/v3/projects => generated 361 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:43.438225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.450857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b025d6-44f4-4dd3-8630-8965d4e869c6 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.528216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fe203-85c9-431d-8ccc-426d2bb13d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.528908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe203-85c9-431d-8ccc-426d2bb13d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.529290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe203-85c9-431d-8ccc-426d2bb13d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.529632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe203-85c9-431d-8ccc-426d2bb13d96 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.530372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fe203-85c9-431d-8ccc-426d2bb13d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.532482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fe203-85c9-431d-8ccc-426d2bb13d96 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.619149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1021/2051] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:43 2022] POST /identity/v3/users => generated 281 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:43.620404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.632688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ccc8c-88a4-4fd9-96e7-32f9fbe53989 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.639048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cf250-fd5a-41a4-b5d8-ee91b2a09498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.639828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf250-fd5a-41a4-b5d8-ee91b2a09498 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.640367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf250-fd5a-41a4-b5d8-ee91b2a09498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.641965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08250da-aefc-4761-b5f7-4691a1aadc2c None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.642304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08250da-aefc-4761-b5f7-4691a1aadc2c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.646483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08250da-aefc-4761-b5f7-4691a1aadc2c None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.646483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1022/2052] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:40:43.646649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf250-fd5a-41a4-b5d8-ee91b2a09498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.646649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf250-fd5a-41a4-b5d8-ee91b2a09498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.647682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf250-fd5a-41a4-b5d8-ee91b2a09498 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.658434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1031/2053] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:43.664084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08250da-aefc-4761-b5f7-4691a1aadc2c None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.679239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.679527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.679800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.684268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08250da-aefc-4761-b5f7-4691a1aadc2c None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.693165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.694092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.694508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.694880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.700415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.702638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.707262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] MFA Rules not processed for user `ce4b77895c73479c9811fa2fd6d7ed71`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:43.709010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:43.743986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1023/2054] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:43.745113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.757845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:43.761565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a7eec-f640-4829-9fd8-85a2770707c5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.766803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64590a4c-6ce1-4aff-8ceb-ea618b709391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.767790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64590a4c-6ce1-4aff-8ceb-ea618b709391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.768820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64590a4c-6ce1-4aff-8ceb-ea618b709391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.769376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64590a4c-6ce1-4aff-8ceb-ea618b709391 admin admin] PATH_INFO: `/v3/projects/2f31c28caabb4a1cbe5d0d5320ca0ad8/users/cbedb8d08a5f4122b83b12dc29a175f5/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.778606 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:43.788917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64590a4c-6ce1-4aff-8ceb-ea618b709391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f31c28caabb4a1cbe5d0d5320ca0ad8, user_id=cbedb8d08a5f4122b83b12dc29a175f5, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.795664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64590a4c-6ce1-4aff-8ceb-ea618b709391 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.797941 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1032/2055] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:40:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:43.810988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.812395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:43.832949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838ef2-f670-4fcb-96c2-bf03a73cce09 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.839955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.840122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.840350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.840609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.854711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.856427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.868945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1033/2056] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:43.869901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:43.871271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1024/2057] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:43 2022] PUT /identity/v3/projects/2f31c28caabb4a1cbe5d0d5320ca0ad8/users/cbedb8d08a5f4122b83b12dc29a175f5/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:43.873534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:43.883191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.883191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.883501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.910123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f289a-e4ce-40f4-80ee-2fb924fc614c service placement] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:43.920429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] MFA Rules not processed for user `cbedb8d08a5f4122b83b12dc29a175f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:43.921710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:43.983007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:43.983584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:43.983863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:43.984145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] PATH_INFO: `/v3/users/7cde8c3305c84856835c9a7f11227157` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:43.987707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cde8c3305c84856835c9a7f11227157)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:43.992768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:43.997351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1025/2058] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:44.015056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:44.015056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:44.070325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d51161-47fc-456a-8a2b-f425c2dadc55 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:44.073983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:44.074588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:44.074863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:44.075130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:44.083508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:44.084979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:44.095171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1026/2059] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:44.095171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:44.097008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:44.119026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86a0f0-4f7f-4c31-b50d-88535d541736 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:44.125174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:44.125995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:44.126349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:44.126698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] PATH_INFO: `/v3/users/7d478c4894c14ae8988ffee14faac23f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:44.128375 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d478c4894c14ae8988ffee14faac23f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:44.130721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:44.229783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cde8c3305c84856835c9a7f11227157'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.230555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cde8c3305c84856835c9a7f11227157'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.231423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cde8c3305c84856835c9a7f11227157'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.248282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cde8c3305c84856835c9a7f11227157'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.297274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d478c4894c14ae8988ffee14faac23f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.306705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d478c4894c14ae8988ffee14faac23f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.307560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d478c4894c14ae8988ffee14faac23f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.322799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d478c4894c14ae8988ffee14faac23f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.324797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:44.337218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-60c270b5-bc99-4a70-b018-2748cf191a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cde8c3305c84856835c9a7f11227157'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.354146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1034/2060] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:43 2022] DELETE /identity/v3/users/7cde8c3305c84856835c9a7f11227157 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:44.412480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d478c4894c14ae8988ffee14faac23f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.423467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1027/2061] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:44 2022] DELETE /identity/v3/users/7d478c4894c14ae8988ffee14faac23f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:44.429246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:44.429545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:44.429830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:44.451956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:44.453173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:44.498858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:44.529842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1035/2062] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:44.539395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:44.540830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:44.605120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37fda6-42ce-4a86-aebc-ba5ecd63e330 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:44.613181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:44.614306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:44.614818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:44.615693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:44.620505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:44.623078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:44.637649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575d566-01e5-432a-a0f3-bbf162bf4c4d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:44.638810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:44.640350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:44.660117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:44.660117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:44.660843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:44.661353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] PATH_INFO: `/v3/projects/d1f75c2759794117a66f839dc2479912` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:44.661539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1028/2063] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:44.662879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f75c2759794117a66f839dc2479912)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:44.664950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:44.852419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f75c2759794117a66f839dc2479912', 'request_id': 'req-00d751ea-e33c-4753-bf90-d1552badc59f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.859138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f75c2759794117a66f839dc2479912', 'request_id': 'req-00d751ea-e33c-4753-bf90-d1552badc59f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.960015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f75c2759794117a66f839dc2479912', 'request_id': 'req-00d751ea-e33c-4753-bf90-d1552badc59f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.973957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] The token cache is being invalidate because project d1f75c2759794117a66f839dc2479912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:44.974260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.974998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:44.976593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1036/2064] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:44 2022] DELETE /identity/v3/projects/d1f75c2759794117a66f839dc2479912 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:44.980487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:44.993599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64677-8086-49e3-be6e-6e03f80b215a service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:45.070003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:45.070982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:45.071510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:45.072048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] PATH_INFO: `/v3/users/41270e7b9e8a4e039dcac3c6ff00038d` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:45.074784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=41270e7b9e8a4e039dcac3c6ff00038d)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:45.078264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:45.213902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:45.216021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:45.229847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d751ea-e33c-4753-bf90-d1552badc59f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:45.237518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:45.238404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:45.240740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:45.240740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] PATH_INFO: `/v3/projects/add21697989041f4a690b2a6b7303fb5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:45.241535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=add21697989041f4a690b2a6b7303fb5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:45.243843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:45.267236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41270e7b9e8a4e039dcac3c6ff00038d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.278541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41270e7b9e8a4e039dcac3c6ff00038d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.279528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41270e7b9e8a4e039dcac3c6ff00038d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.305823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41270e7b9e8a4e039dcac3c6ff00038d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.368182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add21697989041f4a690b2a6b7303fb5', 'request_id': 'req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.374953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add21697989041f4a690b2a6b7303fb5', 'request_id': 'req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.408639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41270e7b9e8a4e039dcac3c6ff00038d'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.419720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1029/2065] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:44 2022] DELETE /identity/v3/users/41270e7b9e8a4e039dcac3c6ff00038d => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:45.543749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add21697989041f4a690b2a6b7303fb5', 'request_id': 'req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.564386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] The token cache is being invalidate because project add21697989041f4a690b2a6b7303fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:45.564816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.565637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.568131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1037/2066] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:45 2022] DELETE /identity/v3/projects/add21697989041f4a690b2a6b7303fb5 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:45.573169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:45.577532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:45.706808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1dc02-f60d-4f86-b1f1-64a9c52bf3ca admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:45.712629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:45.713520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:45.713797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:45.714367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] PATH_INFO: `/v3/users/891f82868aaa4f8997b1e1f60a7fa9d8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:45.716210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:45.717529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=891f82868aaa4f8997b1e1f60a7fa9d8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:45.719525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:45.856200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:45.858193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:45.923549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '891f82868aaa4f8997b1e1f60a7fa9d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.935807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '891f82868aaa4f8997b1e1f60a7fa9d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.937626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '891f82868aaa4f8997b1e1f60a7fa9d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:45.949482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb99c6-fcb7-4e4b-aba1-bb36c81b7dab admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:45.954838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:45.957701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:45.960532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:45.960532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:45.960532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] PATH_INFO: `/v3/projects/44a76dce2bbb4f3baafaaaa117615919` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:45.960532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a76dce2bbb4f3baafaaaa117615919)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:45.963430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:45.965401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '891f82868aaa4f8997b1e1f60a7fa9d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.045890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ae4a4463-110f-475d-bb3a-fe9915b763f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '891f82868aaa4f8997b1e1f60a7fa9d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.054828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1030/2067] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:45 2022] DELETE /identity/v3/users/891f82868aaa4f8997b1e1f60a7fa9d8 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:46.065973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a76dce2bbb4f3baafaaaa117615919', 'request_id': 'req-876e33dc-c78c-4623-a936-1e53ce8d97db'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.073342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a76dce2bbb4f3baafaaaa117615919', 'request_id': 'req-876e33dc-c78c-4623-a936-1e53ce8d97db'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.132817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a76dce2bbb4f3baafaaaa117615919', 'request_id': 'req-876e33dc-c78c-4623-a936-1e53ce8d97db'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.148322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] The token cache is being invalidate because project 44a76dce2bbb4f3baafaaaa117615919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:46.148892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.149791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-876e33dc-c78c-4623-a936-1e53ce8d97db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:46.156005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1038/2068] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:45 2022] DELETE /identity/v3/projects/44a76dce2bbb4f3baafaaaa117615919 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:46.156945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.162344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.162344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.193645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:46.198727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:46.288986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1031/2069] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:46.298009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.298988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.300075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.342610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:46.344976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:46.360269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1039/2070] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:46.360726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.376942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28f97-f472-43f0-8dd8-915c0fdf4778 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.383453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.384268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.384611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.384889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.385801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:46.386110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.388130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.403542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1032/2071] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:40:46.409755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.420765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf69dc2-615b-48b2-bbe2-bdb547d77c58 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.426238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.426855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.427136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.427408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.428177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.430352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.569828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1040/2072] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:46 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:46.574110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.585457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980943fc-600d-42a6-a108-f8754efd3e5b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.645817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.646480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.646769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.647043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.647841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.649897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.694912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.696690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:46.716577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6b600-b41e-4ece-94a5-1708668d81f5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.716577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.717341 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.717768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.717885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] PATH_INFO: `/v3/projects/2fe3b738afe64bd296bd76c178f139c3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.719096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe3b738afe64bd296bd76c178f139c3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.721762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.746411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1033/2073] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:46 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:46.750537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.765572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93ac45-031e-4c55-b31f-a1e26e5b7821 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.772115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.772885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.773325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.773700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.774521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.776514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.792999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1034/2074] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:40:46.794523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.810379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b514f-786b-4441-a6ea-84c3d2e6ab6d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.816230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.817166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.817565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.817917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.818745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.820517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.834965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1035/2075] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:40:46.836173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:46.860097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f562f0d-afe5-4cb4-a3fa-334c4812e7a1 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:46.876228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f651e56-8c7e-4d40-8994-4afb688fd006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:46.877381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f651e56-8c7e-4d40-8994-4afb688fd006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:46.882438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f651e56-8c7e-4d40-8994-4afb688fd006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:46.882438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f651e56-8c7e-4d40-8994-4afb688fd006 admin admin] PATH_INFO: `/v3/projects/3dfb98bb82d548a8a12ed832eeaca6f5/users/1f84a781e0ad42d68dcd1564413f7231/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:46.902856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f651e56-8c7e-4d40-8994-4afb688fd006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfb98bb82d548a8a12ed832eeaca6f5, user_id=1f84a781e0ad42d68dcd1564413f7231, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:46.914281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f651e56-8c7e-4d40-8994-4afb688fd006 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:46.991719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1036/2076] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:46 2022] PUT /identity/v3/projects/3dfb98bb82d548a8a12ed832eeaca6f5/users/1f84a781e0ad42d68dcd1564413f7231/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:47.006939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.007295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.007688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.012193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe3b738afe64bd296bd76c178f139c3', 'request_id': 'req-2bca3c94-26a9-437c-a172-10ff4eac643f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.019726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe3b738afe64bd296bd76c178f139c3', 'request_id': 'req-2bca3c94-26a9-437c-a172-10ff4eac643f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.038047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] MFA Rules not processed for user `1f84a781e0ad42d68dcd1564413f7231`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:47.041583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.103701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe3b738afe64bd296bd76c178f139c3', 'request_id': 'req-2bca3c94-26a9-437c-a172-10ff4eac643f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.103972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1037/2077] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:47.114974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.116329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.117234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] The token cache is being invalidate because project 2fe3b738afe64bd296bd76c178f139c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:47.117547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.118173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2bca3c94-26a9-437c-a172-10ff4eac643f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.119990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1041/2078] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:46 2022] DELETE /identity/v3/projects/2fe3b738afe64bd296bd76c178f139c3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:47.125152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.125780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.126261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.146998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:47.148930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.182389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8de04-6fb6-420a-a8e1-bf554ecf198f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.186159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.186971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.187473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.187765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.189572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.201960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.203470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab65d46-f07d-466f-8350-5fdb7eb97b8f service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.212669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1038/2079] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:47.213832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1042/2080] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:47.223015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.226691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.226691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.233460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.235171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.248306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b2095-fac7-4a23-aa1c-4fb33c6486bc None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.250142 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:47.251351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.253744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.254390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.254671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.254949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] PATH_INFO: `/v3/users/1f84a781e0ad42d68dcd1564413f7231` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.256183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f84a781e0ad42d68dcd1564413f7231)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.258213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.260043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1039/2081] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:47.264326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.274241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b30ce-e6bd-420a-9ac7-c4f08b8cba10 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.278963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.279541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.279825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.280096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.280795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:47.281099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.282749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.294984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1040/2082] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:40:47.299520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.311055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393936-e7b4-466d-8ab1-aebf570d46be admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.319293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.319922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.320197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.320469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.321143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.323262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.460050 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1041/2083] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/projects => generated 401 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:40:47.464020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.491296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:47.507472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f84a781e0ad42d68dcd1564413f7231'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.508134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f84a781e0ad42d68dcd1564413f7231'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.508747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f84a781e0ad42d68dcd1564413f7231'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.523458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f84a781e0ad42d68dcd1564413f7231'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.530710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a9b2f-0c2e-4ad1-930e-b923fd43b24f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.536442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.537085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.537414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.537698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.538377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.540323 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.611991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-efdcf4e8-7087-46a3-ba04-82cdf3bb4e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f84a781e0ad42d68dcd1564413f7231'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:47.624815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1043/2084] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:47 2022] DELETE /identity/v3/users/1f84a781e0ad42d68dcd1564413f7231 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:47.629859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.630182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.630466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.642132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1042/2085] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/users => generated 301 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:40:47.649657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.649896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:47.651092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.651324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.663627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5908e25-73cd-4e9a-9076-d7225a6946ea admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.710669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1044/2086] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:47.721082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:47.722573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.723515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.723999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:47.724151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.724410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.724677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.725405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.727245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.738889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1043/2087] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:47.742859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.753371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfbcf1-46d9-4dda-9195-f231cf3e6de7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.758879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.763427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.763427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.763427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.763427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.763427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.774253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1044/2088] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:47.778342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:47.781138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c8086-a07c-46bc-8d83-83f8f04ec4cb None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.786210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa20b4-1b85-4c5b-82d9-0c027c5a69bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.786210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa20b4-1b85-4c5b-82d9-0c027c5a69bd service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.786462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa20b4-1b85-4c5b-82d9-0c027c5a69bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.786742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa20b4-1b85-4c5b-82d9-0c027c5a69bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.788528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6896c0-dc96-4bd7-a3a4-7d7aa55ce326 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:47.793314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:47.793957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.794232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.794498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] PATH_INFO: `/v3/projects/a64af965c0c8471daad9f6ea11f37925/users/2c946198049f4bd8a2a6af663ed3dca1/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.796137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa20b4-1b85-4c5b-82d9-0c027c5a69bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.797628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa20b4-1b85-4c5b-82d9-0c027c5a69bd service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.807187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1045/2089] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:47.811108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64af965c0c8471daad9f6ea11f37925, user_id=2c946198049f4bd8a2a6af663ed3dca1, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:47.813824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:47.914899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1045/2090] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:47 2022] PUT /identity/v3/projects/a64af965c0c8471daad9f6ea11f37925/users/2c946198049f4bd8a2a6af663ed3dca1/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:47.925366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:47.925662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:47.925951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:47.954490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] MFA Rules not processed for user `2c946198049f4bd8a2a6af663ed3dca1`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:47.956543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.036878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1046/2091] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:48.046456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.124477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29bef-cc93-488b-8c68-a1b71afd2466 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.129517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.130138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.130415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.130912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.131596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.133610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.275947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.278196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.285149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1046/2092] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/projects => generated 401 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:48.289312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.306118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5f4c2-f674-4c36-8d98-f36c8a8accaa None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.352911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4ea9d-cecf-4f42-b858-b59b3d4e0e1b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.357873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.361570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.361570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.361570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.361570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.362808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.365168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.365846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.366144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.366426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] PATH_INFO: `/v3/projects/3dfb98bb82d548a8a12ed832eeaca6f5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.367605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfb98bb82d548a8a12ed832eeaca6f5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.369671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.468013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1047/2093] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/users => generated 307 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:40:48.469165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.480600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9697ba3-0705-48fe-8e96-50c925cf9ba0 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.488769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.489421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.489702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.489978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.490669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.492270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.503471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1048/2094] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:48.506593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfb98bb82d548a8a12ed832eeaca6f5', 'request_id': 'req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:48.507556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.512904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfb98bb82d548a8a12ed832eeaca6f5', 'request_id': 'req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:48.516945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a31c5-db0e-42fb-a1ed-377c197242ba admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.572838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.573650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.573926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.574284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] PATH_INFO: `/v3/projects/21bec7ef05d342d89ecf98d95e8e0d0b/users/27f8cc6ad6de41e4806c86ed1dc6306a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.590747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=21bec7ef05d342d89ecf98d95e8e0d0b, user_id=27f8cc6ad6de41e4806c86ed1dc6306a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.593506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.606305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfb98bb82d548a8a12ed832eeaca6f5', 'request_id': 'req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:48.619692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] The token cache is being invalidate because project 3dfb98bb82d548a8a12ed832eeaca6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:48.619983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:48.620630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ee6c50ff-975f-40d4-9eb6-4062ac5ac8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:48.625634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1047/2095] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:48 2022] DELETE /identity/v3/projects/3dfb98bb82d548a8a12ed832eeaca6f5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:48.630846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccad4e-7dd3-4869-8f55-1de7062a8fea None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.631134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccad4e-7dd3-4869-8f55-1de7062a8fea None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.631413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccad4e-7dd3-4869-8f55-1de7062a8fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.651822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f6ccad4e-7dd3-4869-8f55-1de7062a8fea None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:48.653037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f6ccad4e-7dd3-4869-8f55-1de7062a8fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.661072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1048/2096] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:48.668825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.673296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.673296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.689161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:48.691177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.700229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1049/2097] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:48 2022] PUT /identity/v3/projects/21bec7ef05d342d89ecf98d95e8e0d0b/users/27f8cc6ad6de41e4806c86ed1dc6306a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:48.703870 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.706869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.762155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1049/2098] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:48.766403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.773438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db67f1-ba11-49a3-8f07-866f18690f0d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.776395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8e732-2996-47b9-bb52-58269298297b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.778144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.778758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.779034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.779302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.780016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.781014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.781682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.781775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.781964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.782240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.783065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:48.783372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.785345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.791804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1050/2099] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:40:48.795732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.800015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1050/2100] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:40:48.806726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.808693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd517f8d-34c4-4f25-8352-72a26d0c19a5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.814029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87201cf3-e2e0-46c5-8601-6e1d951825dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.814707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87201cf3-e2e0-46c5-8601-6e1d951825dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.815563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87201cf3-e2e0-46c5-8601-6e1d951825dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.817051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87201cf3-e2e0-46c5-8601-6e1d951825dd admin admin] PATH_INFO: `/v3/projects/21bec7ef05d342d89ecf98d95e8e0d0b/users/27f8cc6ad6de41e4806c86ed1dc6306a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.817978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87201cf3-e2e0-46c5-8601-6e1d951825dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=21bec7ef05d342d89ecf98d95e8e0d0b, user_id=27f8cc6ad6de41e4806c86ed1dc6306a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.820577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab4a2e-b092-43f0-b957-ff786d2cfa8c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.820725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87201cf3-e2e0-46c5-8601-6e1d951825dd admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.825607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.826216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.826496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.826767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.827446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.829534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:48.833934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1051/2101] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:48 2022] PUT /identity/v3/projects/21bec7ef05d342d89ecf98d95e8e0d0b/users/27f8cc6ad6de41e4806c86ed1dc6306a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:40:48.843138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.843421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.843694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.872004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] MFA Rules not processed for user `27f8cc6ad6de41e4806c86ed1dc6306a`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:48.873313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.905188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:48.914550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1051/2102] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/projects => generated 359 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:48.915624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.946074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1052/2103] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:48.959148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:48.960615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:48.979366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06489bfb-613b-4d8a-b3d3-7dde8422ab21 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:48.984129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:48.984746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:48.985031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:48.985342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:48.986009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:48.988293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.017457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02d2bf-7063-4c51-b704-e8ce777d8444 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.068689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.073557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.073557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.073557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.131106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.132586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.133238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1052/2104] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:48 2022] POST /identity/v3/users => generated 280 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:40:49.137696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:49.138894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:49.143158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1053/2105] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:48 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:49.149647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8e6c0-bd96-4e54-913a-522c182baa28 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.155006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.155628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.156127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.156202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.156860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.159440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.171391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1053/2106] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:49.176916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:49.186979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b02ce-16ef-41e4-800c-5af472c4e76e service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.191794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070fd9f-d301-4440-9ba7-c8d023292a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.192480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070fd9f-d301-4440-9ba7-c8d023292a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.192725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070fd9f-d301-4440-9ba7-c8d023292a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.193024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070fd9f-d301-4440-9ba7-c8d023292a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.193789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070fd9f-d301-4440-9ba7-c8d023292a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.195406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070fd9f-d301-4440-9ba7-c8d023292a8e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.206235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1054/2107] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:49.210518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:49.220652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a13d2-94b0-41cc-b567-9c431b79b3d5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.231691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.231691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.231691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.231691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] PATH_INFO: `/v3/projects/8ae68217edbe4943b7489432553d0a00/users/ec7f43c69e4f4ffebfad19910110ac26/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.244915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae68217edbe4943b7489432553d0a00, user_id=ec7f43c69e4f4ffebfad19910110ac26, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.247885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.326779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1054/2108] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:49 2022] PUT /identity/v3/projects/8ae68217edbe4943b7489432553d0a00/users/ec7f43c69e4f4ffebfad19910110ac26/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:49.338249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.338526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.338805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.370030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] MFA Rules not processed for user `ec7f43c69e4f4ffebfad19910110ac26`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:49.371259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:49.375866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:49.434210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371b69b-aa93-47c5-8cc1-118eeb95cde3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.439235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.442731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.442731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.442731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.442731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:49.444570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1055/2109] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:49.485967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:49.488585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:49.500777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf90d3a-864e-46ea-8cac-a6e14bca4619 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.504882 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.505536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.505803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.506072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.516480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.517944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.518770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.520262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e712a-0197-4ded-9a20-ff8345f2576f service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.528078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1056/2110] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:40:49.529837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1055/2111] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:49 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:49.545545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:49.557608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:49.617709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbfa97-4ffc-49f0-9208-11027d86f4ed service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:49.622593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:49.626182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.626182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.626182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] PATH_INFO: `/v3/users/ec7f43c69e4f4ffebfad19910110ac26` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:49.626182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7f43c69e4f4ffebfad19910110ac26)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:49.627570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:49.692106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:49.771878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:49.848627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7f43c69e4f4ffebfad19910110ac26'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:49.856526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7f43c69e4f4ffebfad19910110ac26'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:49.857348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7f43c69e4f4ffebfad19910110ac26'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:49.871373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7f43c69e4f4ffebfad19910110ac26'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:49.968262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7f43c69e4f4ffebfad19910110ac26'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:49.975991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1057/2112] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:40:49 2022] DELETE /identity/v3/users/ec7f43c69e4f4ffebfad19910110ac26 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:49.982353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:49.982648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:49.982926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:50.001996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:50.003763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:50.064581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1056/2113] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:50.075463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:50.076908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:50.133476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a4fcd-3d73-4bd7-a44d-501801ad37b2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:50.137281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dfcf6-0942-43a7-abfd-a2444bdee410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:50.137910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dfcf6-0942-43a7-abfd-a2444bdee410 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:50.138315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dfcf6-0942-43a7-abfd-a2444bdee410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:50.138617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dfcf6-0942-43a7-abfd-a2444bdee410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:50.148492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dfcf6-0942-43a7-abfd-a2444bdee410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:50.149974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dfcf6-0942-43a7-abfd-a2444bdee410 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:50.160627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1058/2114] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:40:50.166865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:50.292679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:50.550112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:50.665411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:50.666874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:50.681107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991fc8e-5c87-4598-9fb7-6a22c9ddbacf None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:50.686383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:50.694825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:50.694825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:50.694825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] PATH_INFO: `/v3/projects/8ae68217edbe4943b7489432553d0a00` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:50.694825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae68217edbe4943b7489432553d0a00)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:50.695915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:50.841090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae68217edbe4943b7489432553d0a00', 'request_id': 'req-6864cdf9-f1c0-4467-a795-aca873cee78f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:50.848168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae68217edbe4943b7489432553d0a00', 'request_id': 'req-6864cdf9-f1c0-4467-a795-aca873cee78f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:50.926683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae68217edbe4943b7489432553d0a00', 'request_id': 'req-6864cdf9-f1c0-4467-a795-aca873cee78f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:50.934310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:50.939507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] The token cache is being invalidate because project 8ae68217edbe4943b7489432553d0a00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:40:50.939798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:50.940545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6864cdf9-f1c0-4467-a795-aca873cee78f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:40:50.942384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1057/2115] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:40:50 2022] DELETE /identity/v3/projects/8ae68217edbe4943b7489432553d0a00 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:50.947353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:50.948136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:50.948423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:50.967923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:50.969135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:51.021263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:40:51.033615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1059/2116] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:40:51.043264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.043632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.043980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.064058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:51.065485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:51.073683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1058/2117] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:51.082251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.089253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2dfdf-a666-4fec-9acd-5c168a4f6c46 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.099944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.100499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.100777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.101046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.101805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:40:51.102105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.103697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.116510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1060/2118] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:40:51.121151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.131214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dc828-f458-4d21-b747-7bec52504557 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.136642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.139731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.139731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.139731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.139731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.140422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.270995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1059/2119] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:40:51 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:51.275169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.285569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3970-38dd-4c72-b7f2-479e62880bcf admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.344751 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.345402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.345778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.345976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.347629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.349598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.412320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1061/2120] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:40:51 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:40:51.416655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.431729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af886dee-d0c1-4f08-bfbf-0ea284320a40 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.436370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.436984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.437302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.437584 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.438314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.439960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.449863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1060/2121] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:51.453908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.464277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8eae0-7a6f-4f9a-9610-c2a7a105f94f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.468966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce196056-3695-4373-a7b9-4c7789cad87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.469607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196056-3695-4373-a7b9-4c7789cad87e admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.469893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196056-3695-4373-a7b9-4c7789cad87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.470186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196056-3695-4373-a7b9-4c7789cad87e admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.470880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196056-3695-4373-a7b9-4c7789cad87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.473096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196056-3695-4373-a7b9-4c7789cad87e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.483770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1062/2122] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:40:51.488812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.498676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96125e05-ad6a-44b3-a48c-9d77a352fd88 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.503626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.504064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.504347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.504626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] PATH_INFO: `/v3/projects/8b7a98a639d14e4dabf56afd8950436d/users/3faa395a20f849a299a0fd2ac5857489/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.520090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7a98a639d14e4dabf56afd8950436d, user_id=3faa395a20f849a299a0fd2ac5857489, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.522837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.577805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1061/2123] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:40:51 2022] PUT /identity/v3/projects/8b7a98a639d14e4dabf56afd8950436d/users/3faa395a20f849a299a0fd2ac5857489/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:40:51.587587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.587879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.588154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.617670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] MFA Rules not processed for user `3faa395a20f849a299a0fd2ac5857489`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:40:51.618799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:51.687996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1063/2124] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:40:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:40:51.699656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:40:51.701451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:40:51.761003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af51a2a-36a7-44a4-9e74-f6e519e7597f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:40:51.764635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e019b5-502e-4283-b630-3bbe5e43e152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:40:51.765273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e019b5-502e-4283-b630-3bbe5e43e152 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:40:51.765557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e019b5-502e-4283-b630-3bbe5e43e152 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:40:51.765835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e019b5-502e-4283-b630-3bbe5e43e152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:40:51.774763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e019b5-502e-4283-b630-3bbe5e43e152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:40:51.776202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e019b5-502e-4283-b630-3bbe5e43e152 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:40:51.786018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1062/2125] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:40:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:01.684958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:01.780958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:03.688186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:03.690295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:03.759280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3ffe2-512d-4c79-a4b5-59b8032918d3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:03.763763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:03.764400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:03.764686 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:03.764974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] PATH_INFO: `/v3/users/2c946198049f4bd8a2a6af663ed3dca1` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:03.766181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c946198049f4bd8a2a6af663ed3dca1)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:03.768173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:03.979714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c946198049f4bd8a2a6af663ed3dca1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:04.023316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c946198049f4bd8a2a6af663ed3dca1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:04.024070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c946198049f4bd8a2a6af663ed3dca1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:04.039700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c946198049f4bd8a2a6af663ed3dca1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:04.108719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c946198049f4bd8a2a6af663ed3dca1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:04.116160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1064/2126] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:03 2022] DELETE /identity/v3/users/2c946198049f4bd8a2a6af663ed3dca1 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:04.120474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:04.120941 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:04.121043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:04.140925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:04.142227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:04.202347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1063/2127] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:04.211898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:04.213358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:04.272263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9d490d-ab2c-4c69-a74b-d8a4410a3c60 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:04.275822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:04.276431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:04.276708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:04.276975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:04.286907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:04.288467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:04.298218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1065/2128] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:04.829781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:04.836759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:04.843202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22325206-35df-43b8-9a3f-b5a7e66a893d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:04.848680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:04.849361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:04.849649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:04.849928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] PATH_INFO: `/v3/projects/a64af965c0c8471daad9f6ea11f37925` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:04.851225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a64af965c0c8471daad9f6ea11f37925)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:04.857439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:04.984370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a64af965c0c8471daad9f6ea11f37925', 'request_id': 'req-ae441942-6e60-4ee3-889e-6d23de1c580d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:04.991851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a64af965c0c8471daad9f6ea11f37925', 'request_id': 'req-ae441942-6e60-4ee3-889e-6d23de1c580d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.079708 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a64af965c0c8471daad9f6ea11f37925', 'request_id': 'req-ae441942-6e60-4ee3-889e-6d23de1c580d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.092441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] The token cache is being invalidate because project a64af965c0c8471daad9f6ea11f37925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:05.092738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.093423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.098806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1064/2129] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:04 2022] DELETE /identity/v3/projects/a64af965c0c8471daad9f6ea11f37925 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:05.099682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:05.112006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115332b8-219a-4598-8a15-1248c6687adf service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:05.134112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:05.188334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:05.188974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:05.189291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:05.189683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] PATH_INFO: `/v3/users/27f8cc6ad6de41e4806c86ed1dc6306a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:05.190769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f8cc6ad6de41e4806c86ed1dc6306a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:05.192741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:05.239025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:05.240685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:05.254624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae441942-6e60-4ee3-889e-6d23de1c580d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:05.259382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:05.260016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:05.260298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:05.260572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] PATH_INFO: `/v3/users/14642b529b214adb97178f23e5e3503d` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:05.261768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14642b529b214adb97178f23e5e3503d)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:05.263698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:05.379078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f8cc6ad6de41e4806c86ed1dc6306a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.386280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f8cc6ad6de41e4806c86ed1dc6306a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.386940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f8cc6ad6de41e4806c86ed1dc6306a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.399811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f8cc6ad6de41e4806c86ed1dc6306a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.442007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14642b529b214adb97178f23e5e3503d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.442691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14642b529b214adb97178f23e5e3503d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.443284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14642b529b214adb97178f23e5e3503d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.451201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7e79a74f-8a40-4526-88e2-8d9ae846058c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f8cc6ad6de41e4806c86ed1dc6306a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.456070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14642b529b214adb97178f23e5e3503d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.457559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1066/2130] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:05 2022] DELETE /identity/v3/users/27f8cc6ad6de41e4806c86ed1dc6306a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:05.514665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14642b529b214adb97178f23e5e3503d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:05.521936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:05.531151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1065/2131] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:05 2022] DELETE /identity/v3/users/14642b529b214adb97178f23e5e3503d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:05.536028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:05.536308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:05.536586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:05.556284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:05.557540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:05.620540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1067/2132] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:05.633885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:05.635557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:05.702803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af2dcc-1668-4682-b596-505fe0800aa0 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:05.706535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911ac24-3a20-409f-97c0-269e3ec80fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:05.707188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911ac24-3a20-409f-97c0-269e3ec80fff service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:05.707473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911ac24-3a20-409f-97c0-269e3ec80fff service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:05.707742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911ac24-3a20-409f-97c0-269e3ec80fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:05.720389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911ac24-3a20-409f-97c0-269e3ec80fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:05.722325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911ac24-3a20-409f-97c0-269e3ec80fff service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:05.733234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1066/2133] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:05.947715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:05.949137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:05.961819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c6b72-259a-4dc8-b8eb-166d2d75b42c None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:05.967217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:05.967872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:05.968157 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:05.968439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] PATH_INFO: `/v3/projects/21bec7ef05d342d89ecf98d95e8e0d0b` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:05.969782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21bec7ef05d342d89ecf98d95e8e0d0b)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:05.973037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.083446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:06.091755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21bec7ef05d342d89ecf98d95e8e0d0b', 'request_id': 'req-b186eae2-c54e-48ec-bb24-518c7675d943'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.097797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21bec7ef05d342d89ecf98d95e8e0d0b', 'request_id': 'req-b186eae2-c54e-48ec-bb24-518c7675d943'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.217090 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21bec7ef05d342d89ecf98d95e8e0d0b', 'request_id': 'req-b186eae2-c54e-48ec-bb24-518c7675d943'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.228704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] The token cache is being invalidate because project 21bec7ef05d342d89ecf98d95e8e0d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:06.229061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.229844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b186eae2-c54e-48ec-bb24-518c7675d943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.231808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1068/2134] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:05 2022] DELETE /identity/v3/projects/21bec7ef05d342d89ecf98d95e8e0d0b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:06.236843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.237222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.237507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.258501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:06.259718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:06.324199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1067/2135] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:06.332422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.332844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.333226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.355076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:06.356396 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:06.366979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.368734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:06.373757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1069/2136] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:06.374896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.382609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1711e2a-3cf0-4566-b9ee-c053805e537a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.387135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff2525-ff83-4b2d-adb5-9f67019235bd None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.387812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.388674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.391581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.391581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] PATH_INFO: `/v3/projects/e3013ec19294499c8b6411b149c17817` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.391581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3013ec19294499c8b6411b149c17817)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.393582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.394743 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.395421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.395760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.396092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.396809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:06.397117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.399596 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.417899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1070/2137] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:41:06.422310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.434531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689ddfc-6977-4539-9679-84ad3229c69a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.439593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.442803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.442803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.442803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.442803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.443926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.566652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3013ec19294499c8b6411b149c17817', 'request_id': 'req-6492714f-0585-4101-b762-2d7112b6fda1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.572951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3013ec19294499c8b6411b149c17817', 'request_id': 'req-6492714f-0585-4101-b762-2d7112b6fda1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.607540 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1071/2138] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:06 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:06.611831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.657011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3013ec19294499c8b6411b149c17817', 'request_id': 'req-6492714f-0585-4101-b762-2d7112b6fda1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.670053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] The token cache is being invalidate because project e3013ec19294499c8b6411b149c17817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:06.670342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.671057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.676431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1068/2139] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:06 2022] DELETE /identity/v3/projects/e3013ec19294499c8b6411b149c17817 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:06.676992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.678818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:06.680369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec157324-ca17-4a47-a05a-bc266dd8543d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.687543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.688168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.688453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.688728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.689431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.691526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.709145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:06.750084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492714f-0585-4101-b762-2d7112b6fda1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.754972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.755614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.755902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.756208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] PATH_INFO: `/v3/users/f80cbb2051af4224ad2a48c28ffde066` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.757524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80cbb2051af4224ad2a48c28ffde066)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.759535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.795208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1072/2140] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:06 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:06.799691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.801349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:06.813441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d3ca1-235e-46d9-9215-339f953dd55c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.819282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.819911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.820192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.820514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.821248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.822934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.837364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1073/2141] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:06.837813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.849995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbdf1b-6993-4100-ba73-6eb8f95013bc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.855302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.855888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.856228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.856507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.857209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.858859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.869591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1074/2142] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:06.873499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:06.882886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84747d6c-02b2-434f-a3b8-23663405c24b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:06.887503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41438e-772e-44b4-9e0e-3305f521d8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:06.888113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41438e-772e-44b4-9e0e-3305f521d8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.888416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41438e-772e-44b4-9e0e-3305f521d8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.893822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41438e-772e-44b4-9e0e-3305f521d8b3 admin admin] PATH_INFO: `/v3/projects/38263c4231e149bbb7160cf8b5d216cd/users/3b49a0d84aad4f38a9e3af98880f8bcc/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:06.903655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41438e-772e-44b4-9e0e-3305f521d8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38263c4231e149bbb7160cf8b5d216cd, user_id=3b49a0d84aad4f38a9e3af98880f8bcc, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:06.906364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41438e-772e-44b4-9e0e-3305f521d8b3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:06.929760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80cbb2051af4224ad2a48c28ffde066'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.929760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80cbb2051af4224ad2a48c28ffde066'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.930212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80cbb2051af4224ad2a48c28ffde066'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.942697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80cbb2051af4224ad2a48c28ffde066'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:06.983952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1075/2143] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:06 2022] PUT /identity/v3/projects/38263c4231e149bbb7160cf8b5d216cd/users/3b49a0d84aad4f38a9e3af98880f8bcc/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:06.994831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:06.995117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:06.995389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.017732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80cbb2051af4224ad2a48c28ffde066'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:07.029694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] MFA Rules not processed for user `3b49a0d84aad4f38a9e3af98880f8bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:07.033094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:07.033245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1069/2144] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:06 2022] DELETE /identity/v3/users/f80cbb2051af4224ad2a48c28ffde066 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:07.046744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:07.112524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1076/2145] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:07.121684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.121684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:07.200441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ad3d-21e6-41e7-9a3c-8dc151f65d59 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.205550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.206206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.206489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.206765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.207537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.209489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.298313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1070/2146] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:07 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:07.302369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.315089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a21cc6-9994-4f84-a5a6-b8f4cacce62f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.382384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.383152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.383429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.384111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.384404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.386403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.476112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1077/2147] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:07 2022] POST /identity/v3/users => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:07.478960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.490226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb720028-5064-4610-b642-2043256e7e37 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.495598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.496130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.496413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.496689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.497422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.499043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.512165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1071/2148] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:07.520269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.530520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee746c-7738-4fe6-9789-8cfe26d32177 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.535125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.535786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.536121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.536339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] PATH_INFO: `/v3/projects/cadc311772b34f9c8e723cdf5807a33d/users/77719cf6e9e5440faca78631b3e4d42f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.551452 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cadc311772b34f9c8e723cdf5807a33d, user_id=77719cf6e9e5440faca78631b3e4d42f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.554732 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.635605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.637452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:07.663179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5b28b-c643-4316-85e8-6cc0a635d7e2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.663358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1078/2149] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:07 2022] PUT /identity/v3/projects/cadc311772b34f9c8e723cdf5807a33d/users/77719cf6e9e5440faca78631b3e4d42f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:07.664029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.670969 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:07.719731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.720471 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.720823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.721165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] PATH_INFO: `/v3/projects/f516683cd3344c45adbdf92d448060c4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.722526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f516683cd3344c45adbdf92d448060c4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.724491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.728562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d74941-11f4-4a39-b505-81054b771a08 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.733602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.734202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.734534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.734809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.741674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.741674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.751602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1079/2150] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:07.752622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.762895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2b8d-12cd-4ee8-8496-1f786fbcc777 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.767685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d6644-696f-453f-8a2f-896ceee17e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.768327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d6644-696f-453f-8a2f-896ceee17e02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.768610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d6644-696f-453f-8a2f-896ceee17e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.768885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d6644-696f-453f-8a2f-896ceee17e02 admin admin] PATH_INFO: `/v3/projects/cadc311772b34f9c8e723cdf5807a33d/users/77719cf6e9e5440faca78631b3e4d42f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.770993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d6644-696f-453f-8a2f-896ceee17e02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cadc311772b34f9c8e723cdf5807a33d, user_id=77719cf6e9e5440faca78631b3e4d42f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:07.773810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d6644-696f-453f-8a2f-896ceee17e02 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:07.784196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1080/2151] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:07 2022] PUT /identity/v3/projects/cadc311772b34f9c8e723cdf5807a33d/users/77719cf6e9e5440faca78631b3e4d42f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:07.795493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.795772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.796047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.806007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:07.818362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f516683cd3344c45adbdf92d448060c4', 'request_id': 'req-fa832fcf-b95a-4421-a92b-dd75a4b86f77'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:07.824455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] MFA Rules not processed for user `77719cf6e9e5440faca78631b3e4d42f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:07.824648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f516683cd3344c45adbdf92d448060c4', 'request_id': 'req-fa832fcf-b95a-4421-a92b-dd75a4b86f77'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:07.825701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:07.875707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f516683cd3344c45adbdf92d448060c4', 'request_id': 'req-fa832fcf-b95a-4421-a92b-dd75a4b86f77'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:07.886975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] The token cache is being invalidate because project f516683cd3344c45adbdf92d448060c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:07.887272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:07.887909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-fa832fcf-b95a-4421-a92b-dd75a4b86f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:07.889632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1072/2152] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:07 2022] DELETE /identity/v3/projects/f516683cd3344c45adbdf92d448060c4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:07.894353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed975f1-1195-4dd6-a67d-ee71ff250c23 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.895210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed975f1-1195-4dd6-a67d-ee71ff250c23 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.895679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed975f1-1195-4dd6-a67d-ee71ff250c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.908658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1081/2153] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:07.919330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1ed975f1-1195-4dd6-a67d-ee71ff250c23 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:07.920551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1ed975f1-1195-4dd6-a67d-ee71ff250c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:07.920761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:07.922443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:07.988945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee776052-8b24-4c54-a80b-280f5a2f0dc8 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:07.990069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1073/2154] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:07.994014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:07.994977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:07.995430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:07.995833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:07.999018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.006705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.006705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.007575 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:08.018589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.020042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.021940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:08.023425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:08.030144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1082/2155] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:08.033919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1074/2156] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:08.040950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.059235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5fdb-9749-448c-b0c8-ae29ee561134 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.064070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.064752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.065034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.065346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.066048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:08.066355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.068197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.086134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1083/2157] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:41:08.090642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.100229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7560f-8a28-456f-95ec-f9e38f6a1ab4 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.105850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.106680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.106861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.107003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.107814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.111519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.146789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.149988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:08.172228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff782c-ca3d-443a-b827-e455050b90d0 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.182317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.183359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.184238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.184696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.187519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:08.225344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1075/2158] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:08 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:08.227276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.251023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:08.260372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.262069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.272127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1084/2159] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:08.300313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310367eb-e379-4519-80cb-e3ded38ad570 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.306081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.307162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.307734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.308345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.309179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.311668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.422073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1076/2160] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:08 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:08.423394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.434570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa8c7c-3202-494a-8e44-256e3b9b0982 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.439467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.440078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.440359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.440633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.441367 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.442973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.453648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1085/2161] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:08.462320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.472366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b1cdf-5bf0-49c3-8226-285a228fc152 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.478446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0483e-3ffc-48a6-adc1-dccdc65577e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.479072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0483e-3ffc-48a6-adc1-dccdc65577e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.479380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0483e-3ffc-48a6-adc1-dccdc65577e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.486238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0483e-3ffc-48a6-adc1-dccdc65577e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.491836 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0483e-3ffc-48a6-adc1-dccdc65577e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.493589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0483e-3ffc-48a6-adc1-dccdc65577e9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.504989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1077/2162] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:08.509837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.520042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b027d-f83f-4094-982b-c935afd8af79 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.525643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.526376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.526724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.527084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] PATH_INFO: `/v3/projects/c947559e59ce4710a96a3f91fa658362/users/f4dfcfee70eb49078be668f02662c4f2/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.529424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:08.542572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c947559e59ce4710a96a3f91fa658362, user_id=f4dfcfee70eb49078be668f02662c4f2, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.545317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.637953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1086/2163] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:08 2022] PUT /identity/v3/projects/c947559e59ce4710a96a3f91fa658362/users/f4dfcfee70eb49078be668f02662c4f2/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:08.649580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.649821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.650106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.679602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] MFA Rules not processed for user `f4dfcfee70eb49078be668f02662c4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:08.680846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:08.750394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1078/2164] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:08.754859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.818641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637653f2-dd3e-4260-a2dc-1535eaaeea10 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:08.824024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:08.824629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:08.824908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:08.825217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:08.825876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:08.827797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:08.951009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1087/2165] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:08 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:08.955004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:08.965630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239a5a-8721-40b9-9eff-597f149880d7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.023449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.024077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.024358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.024634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.025358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.028359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.121354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1079/2166] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:08 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:09.125365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.136560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992a58d-ef55-43ad-86ee-137d799cc6fc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.141280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.141875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.142155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.142557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.143266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.144903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.156349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1088/2167] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:09.165609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.173470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f1bb3-592c-4621-b1ae-69ba23eb7da1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.178156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45427a4f-71a1-430d-9bb2-ec18ba4738ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.179321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45427a4f-71a1-430d-9bb2-ec18ba4738ca admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.179631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45427a4f-71a1-430d-9bb2-ec18ba4738ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.179906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45427a4f-71a1-430d-9bb2-ec18ba4738ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.187025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45427a4f-71a1-430d-9bb2-ec18ba4738ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.187025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45427a4f-71a1-430d-9bb2-ec18ba4738ca admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.200959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1080/2168] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:09.205101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.215511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cf61e-c52d-480d-bb44-b20638ce7bc4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.220124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.220808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.221089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.221401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] PATH_INFO: `/v3/projects/681caccdf0844d3dae84e4840c52c4ef/users/62add246a3644ea6aa8eaa48f2b01704/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.237415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681caccdf0844d3dae84e4840c52c4ef, user_id=62add246a3644ea6aa8eaa48f2b01704, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.240106 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.291245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1089/2169] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:09 2022] PUT /identity/v3/projects/681caccdf0844d3dae84e4840c52c4ef/users/62add246a3644ea6aa8eaa48f2b01704/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:09.302594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.303052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.303139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.312126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:09.331726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] MFA Rules not processed for user `62add246a3644ea6aa8eaa48f2b01704`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:09.332933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:09.396415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1081/2170] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:09.400889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.461143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ebe2a-7f35-4c2d-b817-54fbd08a0c74 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.465758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.466364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.466644 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.466917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.467586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.469536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.584618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1090/2171] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:09 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:09.588659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.599724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00fef9-fd51-45e6-8c50-0b78bb7b2977 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.674691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.675625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.676058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.676545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.677267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.679249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.766062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1082/2172] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:09 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:09.769994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.781079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed0b04-d194-4565-bcf2-0e0573da6772 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.785664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.786257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.786533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.786801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.787488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.790331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.800897 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1091/2173] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:09.805281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.816231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e14fd7-c5f7-4809-949c-87cf94f5af17 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.822044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.822608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.822916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.823171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] PATH_INFO: `/v3/projects/5b85be529dd44aabaa45736c2c1b0120/users/17b38835f4c248e8a5231badc96f8c74/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.839305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b85be529dd44aabaa45736c2c1b0120, user_id=17b38835f4c248e8a5231badc96f8c74, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.842023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:09.921514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1083/2174] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:09 2022] PUT /identity/v3/projects/5b85be529dd44aabaa45736c2c1b0120/users/17b38835f4c248e8a5231badc96f8c74/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:09.925703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:09.935707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9303a-fa54-48ce-883d-7f3fa0a7eea6 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:09.939388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:09.993400 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca4312-15b4-4106-8c7d-a2ce85b6388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:09.993965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca4312-15b4-4106-8c7d-a2ce85b6388c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:09.994249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca4312-15b4-4106-8c7d-a2ce85b6388c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:09.994527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca4312-15b4-4106-8c7d-a2ce85b6388c admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:09.995227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca4312-15b4-4106-8c7d-a2ce85b6388c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:09.996842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca4312-15b4-4106-8c7d-a2ce85b6388c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:10.008682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1092/2175] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:10.012910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:10.023882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c0c6c-b677-4a17-b671-82ee69910695 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:10.028523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:10.029184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:10.029497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:10.029778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] PATH_INFO: `/v3/projects/5b85be529dd44aabaa45736c2c1b0120/users/17b38835f4c248e8a5231badc96f8c74/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:10.035733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b85be529dd44aabaa45736c2c1b0120, user_id=17b38835f4c248e8a5231badc96f8c74, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:10.044894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:10.049344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1084/2176] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:10 2022] PUT /identity/v3/projects/5b85be529dd44aabaa45736c2c1b0120/users/17b38835f4c248e8a5231badc96f8c74/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:10.059360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:10.059646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:10.059919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:10.088240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] MFA Rules not processed for user `17b38835f4c248e8a5231badc96f8c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:10.089529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:10.156145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1093/2177] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:10.167548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:10.169270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:10.223962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:10.229233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec52d-cb1e-44fc-84cf-1fb9d07983c2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:10.233005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:10.233708 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:10.234011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:10.234242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:10.245186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:10.246734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:10.258641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1085/2178] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:11.229458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:11.239205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa7a12-a4f6-48ac-86dc-b639c5c35d7b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:11.243316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7582d00-e4c7-4fc9-a691-649b1f2a4034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:11.243916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7582d00-e4c7-4fc9-a691-649b1f2a4034 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:11.244197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7582d00-e4c7-4fc9-a691-649b1f2a4034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:11.244471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7582d00-e4c7-4fc9-a691-649b1f2a4034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:11.298963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7582d00-e4c7-4fc9-a691-649b1f2a4034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:11.300402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7582d00-e4c7-4fc9-a691-649b1f2a4034 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:11.310049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1094/2179] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:16.860981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:16.862524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:16.977986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7e4b-50f6-4c91-9b1b-9385d04b17ab service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:16.984711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:16.985682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:16.986609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:16.987050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] PATH_INFO: `/v3/users/3b49a0d84aad4f38a9e3af98880f8bcc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:16.988756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b49a0d84aad4f38a9e3af98880f8bcc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:16.993102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:17.204074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b49a0d84aad4f38a9e3af98880f8bcc'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:17.205314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b49a0d84aad4f38a9e3af98880f8bcc'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:17.206164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b49a0d84aad4f38a9e3af98880f8bcc'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:17.220316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b49a0d84aad4f38a9e3af98880f8bcc'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:17.287754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b49a0d84aad4f38a9e3af98880f8bcc'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:17.318192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1086/2180] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:16 2022] DELETE /identity/v3/users/3b49a0d84aad4f38a9e3af98880f8bcc => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:17.335173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:17.335821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:17.335821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:17.357516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:17.360001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:17.388551 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:17.461407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1095/2181] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:17.472478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:17.473942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:17.557250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65b711-dd16-4bb1-a49e-52526028bb53 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:17.562093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:17.562927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:17.563219 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:17.563495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:17.574646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:17.576167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:17.585852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1087/2182] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:18.075265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:18.075265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:18.080347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc784bc-5a05-4eff-b62d-cebe6bf629d9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:18.086083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:18.087168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:18.087924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:18.088357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] PATH_INFO: `/v3/projects/38263c4231e149bbb7160cf8b5d216cd` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:18.089929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] RBAC: Authorizing `identity:delete_project(project_id=38263c4231e149bbb7160cf8b5d216cd)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:18.092707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:18.231885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38263c4231e149bbb7160cf8b5d216cd', 'request_id': 'req-dba7e29e-605d-40d9-bcab-f9dadfea531b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.240130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38263c4231e149bbb7160cf8b5d216cd', 'request_id': 'req-dba7e29e-605d-40d9-bcab-f9dadfea531b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.342815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38263c4231e149bbb7160cf8b5d216cd', 'request_id': 'req-dba7e29e-605d-40d9-bcab-f9dadfea531b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.354994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] The token cache is being invalidate because project 38263c4231e149bbb7160cf8b5d216cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:18.357524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.357524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.357892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1096/2183] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:18 2022] DELETE /identity/v3/projects/38263c4231e149bbb7160cf8b5d216cd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:18.361929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:18.372287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02269a-16a7-40d5-8951-dc704e2fd204 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:18.382306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:18.432049 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:18.432777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:18.433058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:18.433366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] PATH_INFO: `/v3/users/77719cf6e9e5440faca78631b3e4d42f` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:18.434525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77719cf6e9e5440faca78631b3e4d42f)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:18.436469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:18.625215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77719cf6e9e5440faca78631b3e4d42f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.625879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77719cf6e9e5440faca78631b3e4d42f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.626485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77719cf6e9e5440faca78631b3e4d42f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.639971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77719cf6e9e5440faca78631b3e4d42f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.735296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-aaf02d74-635e-46d1-a947-cd5529f90930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77719cf6e9e5440faca78631b3e4d42f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:18.748651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1088/2184] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:18 2022] DELETE /identity/v3/users/77719cf6e9e5440faca78631b3e4d42f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:19.195687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:19.198406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:19.213145 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7e29e-605d-40d9-bcab-f9dadfea531b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:19.272858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:19.273533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:19.273817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:19.274088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] PATH_INFO: `/v3/projects/cadc311772b34f9c8e723cdf5807a33d` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:19.275376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cadc311772b34f9c8e723cdf5807a33d)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:19.277325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:19.417946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cadc311772b34f9c8e723cdf5807a33d', 'request_id': 'req-57135d49-4f03-4863-8032-e0b96cbad521'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:19.419813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:19.424327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cadc311772b34f9c8e723cdf5807a33d', 'request_id': 'req-57135d49-4f03-4863-8032-e0b96cbad521'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:19.522207 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cadc311772b34f9c8e723cdf5807a33d', 'request_id': 'req-57135d49-4f03-4863-8032-e0b96cbad521'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:19.541100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] The token cache is being invalidate because project cadc311772b34f9c8e723cdf5807a33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:19.557928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:19.557928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-57135d49-4f03-4863-8032-e0b96cbad521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:19.557928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1097/2185] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:19 2022] DELETE /identity/v3/projects/cadc311772b34f9c8e723cdf5807a33d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:19.558235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:19.558235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:19.558235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:19.570102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:19.571350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:19.637600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1089/2186] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:19.648337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:19.648836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:19.649754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:19.683345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:19.684690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:19.694193 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1098/2187] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:19.699472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:19.709845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264114fc-74d9-4331-b1e0-6046bc4093c5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:19.716859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:19.730358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:19.730358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:19.730358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:19.731120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:19.731437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:19.740040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:19.764467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1090/2188] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:41:19.769164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:19.784826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd2d56-ca13-4134-ae9a-452732eea6a9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:19.816935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:19.816935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:19.816935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:19.816935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:19.817554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:19.817554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:19.930189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1099/2189] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:19 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:19.935190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:19.945950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78192e41-c0d5-406a-9968-7e4eec33884f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:19.960525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:20.009934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.010578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.010865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.011157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.011823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.013810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.138581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1091/2190] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:19 2022] POST /identity/v3/users => generated 295 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:20.142622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.153868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa2e19-259a-4a5a-92ea-8189f983f996 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:20.158762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.159383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.159651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.159984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.166719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.168413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.192053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1100/2191] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:20.192909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.223898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a94526-1176-4691-9658-f6320ca57381 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:20.230492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417259d5-488c-4e35-9559-769bbd599bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.231299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417259d5-488c-4e35-9559-769bbd599bea admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.231582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417259d5-488c-4e35-9559-769bbd599bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.231856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417259d5-488c-4e35-9559-769bbd599bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.232557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417259d5-488c-4e35-9559-769bbd599bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.234257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417259d5-488c-4e35-9559-769bbd599bea admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.249767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1092/2192] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:20.253715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.265139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4694c-37b4-43cc-8d3f-da7ccab93692 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:20.270001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.270668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.270953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.272251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] PATH_INFO: `/v3/projects/bdccea8a83bb4538971754dd32a1d825/users/5921e8047cd5426b8a236dbdcc8d215e/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.287490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdccea8a83bb4538971754dd32a1d825, user_id=5921e8047cd5426b8a236dbdcc8d215e, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.290389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.393413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1101/2193] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:20 2022] PUT /identity/v3/projects/bdccea8a83bb4538971754dd32a1d825/users/5921e8047cd5426b8a236dbdcc8d215e/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:20.403997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.404292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.404576 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.434297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] MFA Rules not processed for user `5921e8047cd5426b8a236dbdcc8d215e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:20.435620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:20.501464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1093/2194] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:20.505857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.567963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fb04c-f272-465c-8278-6898a354227c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:20.573572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.574186 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.574465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.574737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.575411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.577347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.718764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1102/2195] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:20 2022] POST /identity/v3/projects => generated 389 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:20.719939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.731022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ed9fc-577f-4d77-b1b8-fbbc7400d4d1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:20.795032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.795693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.795981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.796263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.796943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.799252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.939697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1094/2196] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:20 2022] POST /identity/v3/users => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:20.942973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.954250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37899cea-7ec5-4ce4-8aff-d2c02f300f55 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:20.968043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:20.969029 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:20.969370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:20.969919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:20.970632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:20.972255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:20.984115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1103/2197] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:20.988437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:20.999336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30afa49-5f16-4a18-89ea-46db1db58487 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:21.004238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:21.004976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.005361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.005737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] PATH_INFO: `/v3/projects/2599179e057e4aafb840f4c5b049a624/users/39f9f9129ef04227b99b7a44dc2d505b/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.020006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:21.024277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2599179e057e4aafb840f4c5b049a624, user_id=39f9f9129ef04227b99b7a44dc2d505b, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:21.027820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:21.106716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1095/2198] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:20 2022] PUT /identity/v3/projects/2599179e057e4aafb840f4c5b049a624/users/39f9f9129ef04227b99b7a44dc2d505b/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:21.112970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:21.133866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7b631-2391-4edd-8381-4494124fb300 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:21.191531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:21.202464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e4c64-b15a-4e6a-b7b4-2167a741278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:21.203185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4c64-b15a-4e6a-b7b4-2167a741278a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.203538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4c64-b15a-4e6a-b7b4-2167a741278a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.203884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e4c64-b15a-4e6a-b7b4-2167a741278a admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.204654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e4c64-b15a-4e6a-b7b4-2167a741278a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:21.206365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e4c64-b15a-4e6a-b7b4-2167a741278a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:21.217137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1104/2199] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:21.221587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:21.232052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589abcd8-135a-4dd1-bbba-fe84c83bee86 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:21.236963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:21.237645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.237925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.238229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] PATH_INFO: `/v3/projects/2599179e057e4aafb840f4c5b049a624/users/39f9f9129ef04227b99b7a44dc2d505b/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.240210 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2599179e057e4aafb840f4c5b049a624, user_id=39f9f9129ef04227b99b7a44dc2d505b, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:21.242905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:21.253168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1096/2200] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:21 2022] PUT /identity/v3/projects/2599179e057e4aafb840f4c5b049a624/users/39f9f9129ef04227b99b7a44dc2d505b/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:21.264951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.265273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.265547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.286447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:21.288464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:21.301243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] MFA Rules not processed for user `39f9f9129ef04227b99b7a44dc2d505b`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:21.303133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:21.354846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b639f-1552-4d7d-b24f-26740110f887 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:21.358855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:21.359478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.359757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.360032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.362872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:21.388002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1105/2201] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:21 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:21.428435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:21.429922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:21.441602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1097/2202] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:21.446415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:21.448020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:21.459855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab228d53-62a1-4669-a345-6f88460eaed5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:21.463696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e03af-9f83-4a74-a54c-9dcca37fc74f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:21.466535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e03af-9f83-4a74-a54c-9dcca37fc74f service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.466535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e03af-9f83-4a74-a54c-9dcca37fc74f service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.466535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e03af-9f83-4a74-a54c-9dcca37fc74f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.474057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e03af-9f83-4a74-a54c-9dcca37fc74f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:21.475556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e03af-9f83-4a74-a54c-9dcca37fc74f service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:21.487922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1106/2203] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:21 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:21.635718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:21.698691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83f2ab-eabe-4d7c-93ac-66d860b1e3c3 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:21.703166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:21.706674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:21.706674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:21.706674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] PATH_INFO: `/v3/users/5921e8047cd5426b8a236dbdcc8d215e` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:21.706674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5921e8047cd5426b8a236dbdcc8d215e)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:21.707393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:21.975892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:21.985159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5921e8047cd5426b8a236dbdcc8d215e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:21.986118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5921e8047cd5426b8a236dbdcc8d215e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:21.986732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5921e8047cd5426b8a236dbdcc8d215e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:21.999327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5921e8047cd5426b8a236dbdcc8d215e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.107378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5921e8047cd5426b8a236dbdcc8d215e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.120546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1098/2204] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:21 2022] DELETE /identity/v3/users/5921e8047cd5426b8a236dbdcc8d215e => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:22.125035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:22.125521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:22.125785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:22.146115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:22.147312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:22.218440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1107/2205] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:22.232481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:22.234126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:22.292155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01c290-f401-4053-8bb2-5bee2a9c5dd2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:22.295791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:22.296408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:22.296688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:22.296962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:22.306299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:22.308572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:22.318328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1099/2206] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:22.722006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:22.723398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:22.736376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2e6b1-0f2d-4892-ad9d-4f9b5f3064f0 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:22.741271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:22.741861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:22.742137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:22.742407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] PATH_INFO: `/v3/projects/bdccea8a83bb4538971754dd32a1d825` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:22.744014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdccea8a83bb4538971754dd32a1d825)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:22.746680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:22.860098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdccea8a83bb4538971754dd32a1d825', 'request_id': 'req-ce22a58f-5c66-469b-b8cf-b31606e8dab9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.869278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdccea8a83bb4538971754dd32a1d825', 'request_id': 'req-ce22a58f-5c66-469b-b8cf-b31606e8dab9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.871232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:22.954036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdccea8a83bb4538971754dd32a1d825', 'request_id': 'req-ce22a58f-5c66-469b-b8cf-b31606e8dab9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.965975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] The token cache is being invalidate because project bdccea8a83bb4538971754dd32a1d825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:22.966266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.966900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:22.968443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1108/2207] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:22 2022] DELETE /identity/v3/projects/bdccea8a83bb4538971754dd32a1d825 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:22.972398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:22.982183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbf9fe-c339-41fe-92a1-5a6b89a1a293 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:23.039578 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:23.040604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:23.040900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:23.041180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] PATH_INFO: `/v3/users/39f9f9129ef04227b99b7a44dc2d505b` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:23.042388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=39f9f9129ef04227b99b7a44dc2d505b)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:23.044464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:23.208824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39f9f9129ef04227b99b7a44dc2d505b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.209382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39f9f9129ef04227b99b7a44dc2d505b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.210098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39f9f9129ef04227b99b7a44dc2d505b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.218120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:23.223903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39f9f9129ef04227b99b7a44dc2d505b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.273813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6d25be78-7fa4-4f68-b88c-d749573570ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39f9f9129ef04227b99b7a44dc2d505b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.288194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1100/2208] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:22 2022] DELETE /identity/v3/users/39f9f9129ef04227b99b7a44dc2d505b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:23.688264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:23.690566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:23.703601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22a58f-5c66-469b-b8cf-b31606e8dab9 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:23.762852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:23.763550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:23.763842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:23.764117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] PATH_INFO: `/v3/projects/2599179e057e4aafb840f4c5b049a624` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:23.765309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2599179e057e4aafb840f4c5b049a624)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:23.767258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:23.925804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2599179e057e4aafb840f4c5b049a624', 'request_id': 'req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.932578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2599179e057e4aafb840f4c5b049a624', 'request_id': 'req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:23.999678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2599179e057e4aafb840f4c5b049a624', 'request_id': 'req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:24.010851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] The token cache is being invalidate because project 2599179e057e4aafb840f4c5b049a624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:24.011132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:24.011763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ec4eb0e8-c4ef-4d94-a84d-53c5c4a432fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:24.013517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1109/2209] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:23 2022] DELETE /identity/v3/projects/2599179e057e4aafb840f4c5b049a624 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:24.018136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.018595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.018689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.037882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:24.039071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:24.097664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1101/2210] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:24.105852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.106237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.106518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.126007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:24.127204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:24.135083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1110/2211] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:24.139189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.148806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df75eed-c772-44c1-87c9-4f9f95b6d9fa None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:24.154319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:24.154919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.155202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.155478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.156170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:24.156474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:24.158178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:24.173948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1102/2212] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:41:24.178253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.188835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c0323-854c-497a-b3d5-30eec8cf2b21 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:24.194051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:24.194671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.194953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.195226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.195910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:24.198539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:24.319433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1111/2213] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:24 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:24.324850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.335484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cff7d-3dc8-4209-a6ef-4a943cf58611 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:24.397581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:24.398280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.398556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.398826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.399594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:24.402400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:24.503198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1103/2214] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:24 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:24.507296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.517787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81639391-f0bf-4880-bbf5-3269357c1d99 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:24.522611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:24.527479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.527479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.527479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.527479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:24.527479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:24.536789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1112/2215] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:24.541291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.552073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbeca9-f7aa-4ed7-b094-dbb766f128ee admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:24.557647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f905d-fe1f-4793-a37d-11d9e402d347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:24.558258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f905d-fe1f-4793-a37d-11d9e402d347 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.558543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f905d-fe1f-4793-a37d-11d9e402d347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.558817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f905d-fe1f-4793-a37d-11d9e402d347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.559513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f905d-fe1f-4793-a37d-11d9e402d347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:24.561130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f905d-fe1f-4793-a37d-11d9e402d347 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:24.572568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1104/2216] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:24.576677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.587024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541040-1a72-4617-baf3-b554d6a709f7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:24.593060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:24.593359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.593709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.594059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] PATH_INFO: `/v3/projects/bdc5167877d24fb6b70eb2b6dc04dcbc/users/2c69cd2cf12943f481609cfe2e22f5a5/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.615992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc5167877d24fb6b70eb2b6dc04dcbc, user_id=2c69cd2cf12943f481609cfe2e22f5a5, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:24.616959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:24.747738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1113/2217] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:24 2022] PUT /identity/v3/projects/bdc5167877d24fb6b70eb2b6dc04dcbc/users/2c69cd2cf12943f481609cfe2e22f5a5/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:24.762923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:24.763217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:24.763489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:24.819316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] MFA Rules not processed for user `2c69cd2cf12943f481609cfe2e22f5a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:24.823239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:24.902092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:24.972374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1105/2218] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:24.977590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:24.992939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:25.083306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73142b0f-0f9a-48ab-862f-c4f2a82df9b9 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:25.090332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:25.091046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.091345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.091620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.092339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:25.094382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:25.191430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1114/2219] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:24 2022] POST /identity/v3/projects => generated 381 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:25.195579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:25.207729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b5ebf-421c-4880-948c-58717ba6aa88 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:25.271603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:25.272233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.272520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.272796 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.273509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:25.276164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:25.377015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1106/2220] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:25 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:25.379620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:25.390133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb120c7-7fcf-4189-939a-61ad9334bdb1 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:25.395369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:25.396044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.396331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.396608 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.397441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:25.399256 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:25.411147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1115/2221] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:25.416851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:25.428849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbacc7-8784-4d11-87a9-5b686d408197 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:25.440544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:25.441862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.442337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.442757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] PATH_INFO: `/v3/projects/48a9a8f892754f798192417034318971/users/b533d945ec684a288cc46da0b34654e9/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.467330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a9a8f892754f798192417034318971, user_id=b533d945ec684a288cc46da0b34654e9, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:25.470241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:25.538789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1107/2222] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:25 2022] PUT /identity/v3/projects/48a9a8f892754f798192417034318971/users/b533d945ec684a288cc46da0b34654e9/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:25.542825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:25.552952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f4258-7504-4501-af1d-aa1133455668 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:25.661417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd591e-a321-416b-9d85-e66b095d0d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:25.663329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd591e-a321-416b-9d85-e66b095d0d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.663329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd591e-a321-416b-9d85-e66b095d0d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.663329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd591e-a321-416b-9d85-e66b095d0d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.663597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd591e-a321-416b-9d85-e66b095d0d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:25.666315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd591e-a321-416b-9d85-e66b095d0d47 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:25.680708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1116/2223] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:25.686634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:25.716181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568808f1-d4f9-48f4-a1e4-de7daba383d3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:25.720084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:25.721140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.721647 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.721975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] PATH_INFO: `/v3/projects/48a9a8f892754f798192417034318971/users/b533d945ec684a288cc46da0b34654e9/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.724374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a9a8f892754f798192417034318971, user_id=b533d945ec684a288cc46da0b34654e9, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:25.727543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:25.747907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1108/2224] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:25 2022] PUT /identity/v3/projects/48a9a8f892754f798192417034318971/users/b533d945ec684a288cc46da0b34654e9/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:25.756909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3db0e-e8d7-47ad-b15b-baf15a81feb7 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.760038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3db0e-e8d7-47ad-b15b-baf15a81feb7 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.760393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3db0e-e8d7-47ad-b15b-baf15a81feb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.793640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-61c3db0e-e8d7-47ad-b15b-baf15a81feb7 None None] MFA Rules not processed for user `b533d945ec684a288cc46da0b34654e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:25.794921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-61c3db0e-e8d7-47ad-b15b-baf15a81feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:25.910703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1117/2225] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:25.925627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:25.928064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:25.936019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab4168-630d-40e6-9d50-91e4890197cf None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:25.936019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab4168-630d-40e6-9d50-91e4890197cf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:25.936019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab4168-630d-40e6-9d50-91e4890197cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:25.943639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:25.955154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2bab4168-630d-40e6-9d50-91e4890197cf None None] MFA Rules not processed for user `d8a3eed3ee0b42f69948aced53729d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:25.961499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2bab4168-630d-40e6-9d50-91e4890197cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:26.002366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2e49e-608a-40b3-ba0e-0aacde499e5a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:26.010095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:26.011122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:26.011888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:26.012339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:26.026208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:26.028269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:26.042984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1109/2226] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:26.047733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1118/2227] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:26.799264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:26.848997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:26.904168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de010c-5704-4018-a20a-3d0e446a7728 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:26.908474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:26.909224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:26.909516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:26.909795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] PATH_INFO: `/v3/users/2c69cd2cf12943f481609cfe2e22f5a5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:26.911438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c69cd2cf12943f481609cfe2e22f5a5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:26.913503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:27.244841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c69cd2cf12943f481609cfe2e22f5a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:27.245606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c69cd2cf12943f481609cfe2e22f5a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:27.246282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c69cd2cf12943f481609cfe2e22f5a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:27.264009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c69cd2cf12943f481609cfe2e22f5a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:27.388821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c69cd2cf12943f481609cfe2e22f5a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:27.415913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1110/2228] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:26 2022] DELETE /identity/v3/users/2c69cd2cf12943f481609cfe2e22f5a5 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:27.417895 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:27.418394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:27.418394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:27.446315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:27.446721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:27.530694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1119/2229] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:27.540536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:27.542740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:27.613481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f49c88-7c21-42ad-83a5-5adeb27ea4ac admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:27.617734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:27.618099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:27.618447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:27.618785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:27.628801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:27.630389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:27.644819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1111/2230] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:27.952388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:27.954528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:27.969168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa9571-78b9-4c4b-a972-43f21076a18b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:27.975013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:27.975893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:27.976702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:27.976702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] PATH_INFO: `/v3/users/f4dfcfee70eb49078be668f02662c4f2` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:27.978104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4dfcfee70eb49078be668f02662c4f2)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:27.980725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:28.188700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:28.190244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:28.220893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12962e-c525-44dd-a7a7-91d3924de9b3 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:28.226253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:28.226696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4dfcfee70eb49078be668f02662c4f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.227035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:28.227469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:28.227735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] PATH_INFO: `/v3/projects/bdc5167877d24fb6b70eb2b6dc04dcbc` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:28.229879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc5167877d24fb6b70eb2b6dc04dcbc)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:28.231925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:28.239254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4dfcfee70eb49078be668f02662c4f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.240087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4dfcfee70eb49078be668f02662c4f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.266450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4dfcfee70eb49078be668f02662c4f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.368187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f67d3ff1-6a97-400a-9659-de8ceb45ee77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4dfcfee70eb49078be668f02662c4f2'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.397542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:28.400937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1120/2231] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:27 2022] DELETE /identity/v3/users/f4dfcfee70eb49078be668f02662c4f2 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:28.406358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc5167877d24fb6b70eb2b6dc04dcbc', 'request_id': 'req-907c4040-aa41-4517-8474-4c0afe3b1c1e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.406985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:28.407285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:28.407564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:28.418683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc5167877d24fb6b70eb2b6dc04dcbc', 'request_id': 'req-907c4040-aa41-4517-8474-4c0afe3b1c1e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.456869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:28.470626 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:28.494869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:28.536931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc5167877d24fb6b70eb2b6dc04dcbc', 'request_id': 'req-907c4040-aa41-4517-8474-4c0afe3b1c1e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.554399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] The token cache is being invalidate because project bdc5167877d24fb6b70eb2b6dc04dcbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:28.554855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.557077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.559272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1121/2232] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:28.565526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1112/2233] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:28 2022] DELETE /identity/v3/projects/bdc5167877d24fb6b70eb2b6dc04dcbc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:28.566835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:28.568570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:28.585693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856751-5cb4-455c-92f2-2d24867c0df3 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:28.594752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:28.597984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:28.602351 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:28.605928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:28.606328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:28.606617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] PATH_INFO: `/v3/users/b533d945ec684a288cc46da0b34654e9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:28.610270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b533d945ec684a288cc46da0b34654e9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:28.613989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:28.701335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c4040-aa41-4517-8474-4c0afe3b1c1e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:28.783669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:28.784332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:28.785093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:28.785759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:28.787789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:28.807957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:28.810317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:28.850893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1113/2234] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:28.862484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b533d945ec684a288cc46da0b34654e9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.880149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b533d945ec684a288cc46da0b34654e9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.882061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b533d945ec684a288cc46da0b34654e9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.914619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b533d945ec684a288cc46da0b34654e9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:28.999951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b533d945ec684a288cc46da0b34654e9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:29.007899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1122/2235] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:28 2022] DELETE /identity/v3/users/b533d945ec684a288cc46da0b34654e9 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:29.558630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:29.560603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:29.592054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302eb5f-3c88-4107-99a0-7fe9e8e91ff9 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:29.605564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:29.607078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:29.697670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:29.697670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:29.697670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:29.697670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] PATH_INFO: `/v3/projects/48a9a8f892754f798192417034318971` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:29.699502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a9a8f892754f798192417034318971)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:29.701638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:29.756391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928389c5-7df3-4d80-b499-6a5b6c1e9b99 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:29.768146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:29.769852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:29.770428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:29.770853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] PATH_INFO: `/v3/projects/c947559e59ce4710a96a3f91fa658362` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:29.773039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c947559e59ce4710a96a3f91fa658362)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:29.775749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:29.855389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:29.868426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a9a8f892754f798192417034318971', 'request_id': 'req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:29.879788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a9a8f892754f798192417034318971', 'request_id': 'req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:29.940277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:29.951500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c947559e59ce4710a96a3f91fa658362', 'request_id': 'req-0f365c77-8140-4d06-9632-c489251233f8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:29.962479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c947559e59ce4710a96a3f91fa658362', 'request_id': 'req-0f365c77-8140-4d06-9632-c489251233f8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.005400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a9a8f892754f798192417034318971', 'request_id': 'req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.025920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] The token cache is being invalidate because project 48a9a8f892754f798192417034318971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:30.026262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.027010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1cb7df8a-d232-4616-b1c0-efc93dfc70bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.029397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1114/2236] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:29 2022] DELETE /identity/v3/projects/48a9a8f892754f798192417034318971 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:30.035504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.035845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.036119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.058912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:30.060492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:30.090097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c947559e59ce4710a96a3f91fa658362', 'request_id': 'req-0f365c77-8140-4d06-9632-c489251233f8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.102392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] The token cache is being invalidate because project c947559e59ce4710a96a3f91fa658362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:30.105390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.105390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.106466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1123/2237] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:29 2022] DELETE /identity/v3/projects/c947559e59ce4710a96a3f91fa658362 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:30.107506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:30.108978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:30.144581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1115/2238] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:30.144581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:30.148615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:30.178439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380a956-3951-42e7-b1ce-bc4b85e9cb52 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:30.189497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:30.189497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.189497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.189497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] PATH_INFO: `/v3/users/62add246a3644ea6aa8eaa48f2b01704` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.189946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62add246a3644ea6aa8eaa48f2b01704)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:30.191161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:30.193645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f365c77-8140-4d06-9632-c489251233f8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:30.201567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:30.202284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.202747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.203038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] PATH_INFO: `/v3/users/3faa395a20f849a299a0fd2ac5857489` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.204401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3faa395a20f849a299a0fd2ac5857489)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:30.206386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:30.357501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62add246a3644ea6aa8eaa48f2b01704'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.361160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62add246a3644ea6aa8eaa48f2b01704'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.361160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62add246a3644ea6aa8eaa48f2b01704'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.380572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3faa395a20f849a299a0fd2ac5857489'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.382751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62add246a3644ea6aa8eaa48f2b01704'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.394042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3faa395a20f849a299a0fd2ac5857489'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.397607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3faa395a20f849a299a0fd2ac5857489'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.441475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3faa395a20f849a299a0fd2ac5857489'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.514621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-299d4ac1-2106-4cc7-aead-5377d5d46ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62add246a3644ea6aa8eaa48f2b01704'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.535464 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1116/2239] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:30 2022] DELETE /identity/v3/users/62add246a3644ea6aa8eaa48f2b01704 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:30.538787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.538787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.538787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.565555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:30.567897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:30.571448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-a5935fbd-534a-422e-ac31-7d093079e0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3faa395a20f849a299a0fd2ac5857489'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:30.582431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1124/2240] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:30 2022] DELETE /identity/v3/users/3faa395a20f849a299a0fd2ac5857489 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:30.591873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.591873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.591873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.636346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:30.637549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:30.653512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:30.708017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:30.712077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1117/2241] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:30.719084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:30.729317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10e81f-3b31-438b-8c56-f9628b47ac64 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:30.741296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1125/2242] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:30.754517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:30.757720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:30.802189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:30.803023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.803463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.803832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.804657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:30.805058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:30.807172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:30.833004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1118/2243] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:41:30.837313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:30.856568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4797d-f705-48cf-b036-7e692e92afa2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:30.862634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:30.863278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.863606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.863901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.864769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:30.866823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c685e87-7632-4757-8b41-a54f802af7c5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:30.872202 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:30.875403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:30.876529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:30.876907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:30.877410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:30.892165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:30.893929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:30.906607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1126/2244] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:41:30.992301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1119/2245] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:30 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:30.997862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.012492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b358a-45f6-4466-920d-e136b8391a6a service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.161765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.166560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.166560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.166560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.172644 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.173287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:31.264675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1127/2246] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:30 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:31.270096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.271957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:31.276123 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.308033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3fa9c2-5e6e-4efd-9e92-8ac4271e1ab2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.316222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95980076-1601-409e-a6e5-2af149da41af admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.319025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.319788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.321733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.321733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.321733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.323676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.324572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:31.324972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.325312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.325588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] PATH_INFO: `/v3/projects/681caccdf0844d3dae84e4840c52c4ef` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.326814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681caccdf0844d3dae84e4840c52c4ef)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.328850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:31.341165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1128/2247] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:31.346262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.377484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13758a-d586-43dd-93bd-946d72c9c615 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.388189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.388588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.389156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.389317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.390466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.392150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:31.406419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1129/2248] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:31.410496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.432084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2463a1a-941e-47e3-ada0-32611443283d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.434731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec901ed-23aa-4ed8-891e-46a2f3ee410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.435412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec901ed-23aa-4ed8-891e-46a2f3ee410e admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.435690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec901ed-23aa-4ed8-891e-46a2f3ee410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.435965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec901ed-23aa-4ed8-891e-46a2f3ee410e admin admin] PATH_INFO: `/v3/projects/cf95e3e43d0e482ba4d27b769fb48628/users/dcc12077b11448a3b1ec2e7dbf2707ef/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.454808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec901ed-23aa-4ed8-891e-46a2f3ee410e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf95e3e43d0e482ba4d27b769fb48628, user_id=dcc12077b11448a3b1ec2e7dbf2707ef, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.457648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec901ed-23aa-4ed8-891e-46a2f3ee410e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:31.475621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681caccdf0844d3dae84e4840c52c4ef', 'request_id': 'req-8cb4f664-7a0c-4930-a7cf-625495dca249'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:31.483718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681caccdf0844d3dae84e4840c52c4ef', 'request_id': 'req-8cb4f664-7a0c-4930-a7cf-625495dca249'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:31.582753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681caccdf0844d3dae84e4840c52c4ef', 'request_id': 'req-8cb4f664-7a0c-4930-a7cf-625495dca249'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:31.590849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1130/2249] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:31 2022] PUT /identity/v3/projects/cf95e3e43d0e482ba4d27b769fb48628/users/dcc12077b11448a3b1ec2e7dbf2707ef/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:31.595883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.596172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.596483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.599598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] The token cache is being invalidate because project 681caccdf0844d3dae84e4840c52c4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:31.600177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:31.601063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:31.606630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1120/2250] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:31 2022] DELETE /identity/v3/projects/681caccdf0844d3dae84e4840c52c4ef => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:31.607798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.609812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:31.626860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:31.626860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:31.656520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:31.704102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4f664-7a0c-4930-a7cf-625495dca249 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.704960 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:31.787060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1131/2251] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:31.815973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:31.818861 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:31.835938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa4c47-1f68-40c3-ad07-ebfc7c7b6902 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:31.842793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.843931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.845129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.845606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] PATH_INFO: `/v3/users/17b38835f4c248e8a5231badc96f8c74` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.854720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b38835f4c248e8a5231badc96f8c74)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.854720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:31.861632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:31.862940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:31.863230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:31.863507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] PATH_INFO: `/v3/projects/8b7a98a639d14e4dabf56afd8950436d` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:31.873103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7a98a639d14e4dabf56afd8950436d)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:31.877625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:32.097542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b38835f4c248e8a5231badc96f8c74'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.097661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7a98a639d14e4dabf56afd8950436d', 'request_id': 'req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.097748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b38835f4c248e8a5231badc96f8c74'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.097748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b38835f4c248e8a5231badc96f8c74'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.104228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7a98a639d14e4dabf56afd8950436d', 'request_id': 'req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.124869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b38835f4c248e8a5231badc96f8c74'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.174300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7a98a639d14e4dabf56afd8950436d', 'request_id': 'req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.202659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] The token cache is being invalidate because project 8b7a98a639d14e4dabf56afd8950436d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:32.203537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.204246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.206482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1132/2252] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:31 2022] DELETE /identity/v3/projects/8b7a98a639d14e4dabf56afd8950436d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:32.207211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:32.208976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:32.219179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-ae5336a9-85c3-47a5-8840-cf6be85990ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b38835f4c248e8a5231badc96f8c74'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:32.241892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1121/2253] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:31 2022] DELETE /identity/v3/users/17b38835f4c248e8a5231badc96f8c74 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:32.247037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d6c65-08e5-4cd7-a49a-9f2632f9d1de None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:32.247186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d6c65-08e5-4cd7-a49a-9f2632f9d1de None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:32.249655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d6c65-08e5-4cd7-a49a-9f2632f9d1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:32.298114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c16d6c65-08e5-4cd7-a49a-9f2632f9d1de None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:32.300226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c16d6c65-08e5-4cd7-a49a-9f2632f9d1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:32.339324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ed96-0a48-4aa4-9be1-bb4ef1b9e360 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:32.383126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:32.410293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1122/2254] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:32.421988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:32.423346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:32.423813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:32.443905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:32.445474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:32.445763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:32.446182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:32.448752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:32.488429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:32.494202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:32.504435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:32.506204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:32.519547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1133/2255] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:32.540021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1123/2256] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:32.544808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:32.556917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f93eb4-686f-4138-9740-6efd45112999 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:32.574361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:32.575079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:32.575433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:32.575810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:32.578604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:32.579013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:32.584740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:32.604600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:32.611658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1134/2257] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:41:32.617371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:32.651694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cb266-9297-4916-980f-52713976ba0a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:32.672199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:32.672199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:32.672199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:32.672846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:32.674595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:32.677292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:32.877457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1124/2258] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:32 2022] POST /identity/v3/projects => generated 373 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:32.880776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:32.893166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbd0c6-a302-48b2-b23d-7f5feda7948d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:32.921424 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:32.923343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:32.980104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:32.981312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:32.981649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:32.982685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:32.983580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:32.987696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.032214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b207771-f520-4d6b-8682-1b8c636bbdbe admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.037232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.037886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.038171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.038476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] PATH_INFO: `/v3/users/d9f4252210bd412ba24305b00a6e77b4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.039728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f4252210bd412ba24305b00a6e77b4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.045237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.076078 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1135/2259] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:32 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:33.080883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.096087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254b43a-ef55-43b4-8031-7bb353b75b2b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.101238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.102191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.102442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.102796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.103620 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.105362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.121339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1136/2260] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:33.121339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.121339 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.133868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c79516-ecfe-4518-bd1b-8e4c15c2249f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.139371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.140017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.140304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.140580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] PATH_INFO: `/v3/projects/5b85be529dd44aabaa45736c2c1b0120` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.141784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b85be529dd44aabaa45736c2c1b0120)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.143704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.265733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f4252210bd412ba24305b00a6e77b4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.266505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f4252210bd412ba24305b00a6e77b4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.267222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f4252210bd412ba24305b00a6e77b4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.285042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f4252210bd412ba24305b00a6e77b4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.330543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b85be529dd44aabaa45736c2c1b0120', 'request_id': 'req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.336713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b85be529dd44aabaa45736c2c1b0120', 'request_id': 'req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.365896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f4252210bd412ba24305b00a6e77b4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.379492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1125/2261] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:32 2022] DELETE /identity/v3/users/d9f4252210bd412ba24305b00a6e77b4 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:33.380224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.382206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.451053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b85be529dd44aabaa45736c2c1b0120', 'request_id': 'req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.467933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:33.474913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] The token cache is being invalidate because project 5b85be529dd44aabaa45736c2c1b0120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:33.475799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.476345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b25c1e49-8dab-4ede-a90a-fbc0635d7f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:33.479553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1137/2262] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:33 2022] DELETE /identity/v3/projects/5b85be529dd44aabaa45736c2c1b0120 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:33.486617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4804c-8da8-4dbf-a280-7b9a4cb35108 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.496198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4804c-8da8-4dbf-a280-7b9a4cb35108 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.496198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4804c-8da8-4dbf-a280-7b9a4cb35108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.503154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b9a20-41af-473c-8177-a00744da1f13 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.511449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.512096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.512489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.512890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.513732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.516211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.533330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1126/2263] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:33.538493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.540192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.550802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-13c4804c-8da8-4dbf-a280-7b9a4cb35108 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:33.552101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-13c4804c-8da8-4dbf-a280-7b9a4cb35108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.559485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac0fc9-ed14-4522-92b2-5ae90fc9ed60 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.571160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c934cf-b017-4d0c-a44b-2fa948013927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.574015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c934cf-b017-4d0c-a44b-2fa948013927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.574417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c934cf-b017-4d0c-a44b-2fa948013927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.575071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c934cf-b017-4d0c-a44b-2fa948013927 admin admin] PATH_INFO: `/v3/projects/e5e3d5ca106841cea8f4f07cfc54ca21/users/3b6fe832e8444c2d897eef15d1d2bae6/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.575237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1138/2264] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:33.582035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.582932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.582932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.614875 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c934cf-b017-4d0c-a44b-2fa948013927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e3d5ca106841cea8f4f07cfc54ca21, user_id=3b6fe832e8444c2d897eef15d1d2bae6, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.616938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c934cf-b017-4d0c-a44b-2fa948013927 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.627033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:33.637495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.646124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1139/2265] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:33.650491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.661822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23841433-383a-41b3-8e77-7eaec2e2b999 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.668763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.669163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.669469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.669745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.670659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:33.670916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.672819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.690944 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1140/2266] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:41:33.695737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.706144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04209d-e603-4001-9ed0-3d040e95b2dd admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:33.712003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:33.728741 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1127/2267] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:33 2022] PUT /identity/v3/projects/e5e3d5ca106841cea8f4f07cfc54ca21/users/3b6fe832e8444c2d897eef15d1d2bae6/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:33.729598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.729994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.730274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.731002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:33.732953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:33.740086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:33.740466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:33.740813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:33.788845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] MFA Rules not processed for user `3b6fe832e8444c2d897eef15d1d2bae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:33.790360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.922178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1128/2268] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:33.923581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.925228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:33.942990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1141/2269] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:33 2022] POST /identity/v3/projects => generated 375 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:33.946246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:33.949442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:34.007462 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67487b-c52d-4388-934b-a05088d47789 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.027083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb27856-8605-4b04-bf45-f96f69449d0f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.030955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.031656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.032008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.032358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.083493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.084153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.084493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.084777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] PATH_INFO: `/v3/projects/8a97e25e7eae402f8f5c4222019ed87c` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.086014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a97e25e7eae402f8f5c4222019ed87c)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.088220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.103647 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.107516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.117096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1142/2270] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:34.117887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.130349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487af0e-d565-4794-b56b-7f6680f2f1ef service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.134978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.135720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.135995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.136267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.136936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.138899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.217939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:34.245093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a97e25e7eae402f8f5c4222019ed87c', 'request_id': 'req-76a56d22-eb69-45ad-99a5-595c0062f5c4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:34.254121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a97e25e7eae402f8f5c4222019ed87c', 'request_id': 'req-76a56d22-eb69-45ad-99a5-595c0062f5c4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:34.257063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1143/2271] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:34 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:34.262342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.282279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600e38a-b3af-4972-a0ba-beaa3a015334 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.342847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a97e25e7eae402f8f5c4222019ed87c', 'request_id': 'req-76a56d22-eb69-45ad-99a5-595c0062f5c4'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:34.356722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.357569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.357983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.358389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.360297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.362434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.365031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] The token cache is being invalidate because project 8a97e25e7eae402f8f5c4222019ed87c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:34.365483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:34.366542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-76a56d22-eb69-45ad-99a5-595c0062f5c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:34.368427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1129/2272] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:33 2022] DELETE /identity/v3/projects/8a97e25e7eae402f8f5c4222019ed87c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:34.376381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1144/2273] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:41:34.384392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0eba-48aa-45ea-b4ed-c2b3c5ecd32e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.384392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0eba-48aa-45ea-b4ed-c2b3c5ecd32e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.384392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b0eba-48aa-45ea-b4ed-c2b3c5ecd32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.386803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.388586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:34.407367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-867b0eba-48aa-45ea-b4ed-c2b3c5ecd32e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:34.408354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53ac13f-e5b7-47b1-a208-fff6bec0d009 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.408620 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-867b0eba-48aa-45ea-b4ed-c2b3c5ecd32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:34.413605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.414222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.414507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.414781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.415485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.417158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.417393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1130/2274] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:34.429890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1145/2275] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:34.431921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.432352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.432719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.434721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.446915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097aa84-0888-48ca-91dc-f36664cf26ba admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.453271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341646f-6e17-443a-beb1-802e8adadf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.453978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341646f-6e17-443a-beb1-802e8adadf3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.454263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341646f-6e17-443a-beb1-802e8adadf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.454539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341646f-6e17-443a-beb1-802e8adadf3b admin admin] PATH_INFO: `/v3/projects/415a1461fd1344acb8828e0d78d03717/users/5bd06a639d6e41a3ae3b9d5eb9b7ce79/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.456868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:34.458118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:34.466928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1131/2276] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:34.471329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.480614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341646f-6e17-443a-beb1-802e8adadf3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=415a1461fd1344acb8828e0d78d03717, user_id=5bd06a639d6e41a3ae3b9d5eb9b7ce79, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.480614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341646f-6e17-443a-beb1-802e8adadf3b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.481030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7341b7d-b5f5-441a-bc02-1513749160a8 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.485897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.486500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.486775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.487041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.487733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:34.488031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.489666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.501855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1132/2277] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:41:34.510019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.519803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05938245-f5a2-4135-a3a7-7179a968c927 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.524881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.525564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.525844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.526246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.526766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.528693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.589281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1146/2278] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:34 2022] PUT /identity/v3/projects/415a1461fd1344acb8828e0d78d03717/users/5bd06a639d6e41a3ae3b9d5eb9b7ce79/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:41:34.605815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.605815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.605815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.619581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1133/2279] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:34 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:34.625442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.647234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] MFA Rules not processed for user `5bd06a639d6e41a3ae3b9d5eb9b7ce79`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:34.648583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:34.720700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfa44a-7d6a-45fe-b6da-a44e87e2a2ef admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.726599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1147/2280] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:34.730166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.730815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.731340 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.731630 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.732350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.735980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.748549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.751616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:34.869547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1134/2281] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:34 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:34.873931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.906554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75668b4c-ec35-454b-809d-5f76fdc8a330 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.921135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.922065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.922516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.923068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.923933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.928649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:34.942654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57617979-74cd-4ffb-a21f-427ae35c0f74 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.946456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:34.955860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1135/2282] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:34.956878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:34.957698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:34.959493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:34.959780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:34.981911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:34.982237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d885a-31f8-4425-818c-71c24f37fd9d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:34.984242 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:35.015271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02941947-e8ec-4be5-87fb-cd537bb60ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:35.015831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1148/2283] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:35.016254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02941947-e8ec-4be5-87fb-cd537bb60ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:35.016699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02941947-e8ec-4be5-87fb-cd537bb60ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:35.017121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02941947-e8ec-4be5-87fb-cd537bb60ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:35.018005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02941947-e8ec-4be5-87fb-cd537bb60ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:35.019805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02941947-e8ec-4be5-87fb-cd537bb60ca0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:35.037106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1136/2284] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:35.037560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:35.059236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe0e55-1e18-4120-a0c2-b5738906c364 service glance] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:35.073264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:35.074159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:35.077006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:35.077484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] PATH_INFO: `/v3/projects/274bea7582354f49841b893c0a39ffdf/users/55beb38ea2994ea7987fe546235b58a5/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:35.121227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274bea7582354f49841b893c0a39ffdf, user_id=55beb38ea2994ea7987fe546235b58a5, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:35.125167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:35.255358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1149/2285] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:35 2022] PUT /identity/v3/projects/274bea7582354f49841b893c0a39ffdf/users/55beb38ea2994ea7987fe546235b58a5/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:35.261075 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:35.261803 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:35.262082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:35.269072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:35.298428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:35.299662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:35.451915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1137/2286] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:35.461791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:35.464916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:35.574494 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c7ef2-6fe0-4d2d-b8cc-0bde87fecc99 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:35.580477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38b8e8-7fef-44ee-bb22-6b2e185461d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:35.582738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38b8e8-7fef-44ee-bb22-6b2e185461d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:35.583030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38b8e8-7fef-44ee-bb22-6b2e185461d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:35.583444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38b8e8-7fef-44ee-bb22-6b2e185461d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:35.596026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38b8e8-7fef-44ee-bb22-6b2e185461d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:35.597577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38b8e8-7fef-44ee-bb22-6b2e185461d8 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:35.608173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1150/2287] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:36.120892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:36.453091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:41.058379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:41.059852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:41.092056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14e2cc-1e06-4831-878f-69a3f7c092d0 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:41.107867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:41.108807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:41.109350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:41.109785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] PATH_INFO: `/v3/users/3b6fe832e8444c2d897eef15d1d2bae6` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:41.112512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b6fe832e8444c2d897eef15d1d2bae6)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:41.118230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:41.466161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b6fe832e8444c2d897eef15d1d2bae6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:41.467204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b6fe832e8444c2d897eef15d1d2bae6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:41.468634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b6fe832e8444c2d897eef15d1d2bae6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:41.522155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b6fe832e8444c2d897eef15d1d2bae6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:41.589320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b6fe832e8444c2d897eef15d1d2bae6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:41.615866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1138/2288] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:41:41 2022] DELETE /identity/v3/users/3b6fe832e8444c2d897eef15d1d2bae6 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:41.621898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b053e-468d-4c29-8575-72fd3c65b8a0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:41.622430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b053e-468d-4c29-8575-72fd3c65b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:41.627564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b053e-468d-4c29-8575-72fd3c65b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:41.631706 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:41.658561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2b2b053e-468d-4c29-8575-72fd3c65b8a0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:41.660037 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2b2b053e-468d-4c29-8575-72fd3c65b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:41.778120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1151/2289] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:41.784635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:41.787083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:41.912715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834d2d1-8978-465d-a87e-d50e557bb50c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:41.917166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:41.918023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:41.918411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:41.918789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:41.940214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:41.941928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:41.962688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1139/2290] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:42.252605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:42.253041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:42.253591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:42.298148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] MFA Rules not processed for user `dcc12077b11448a3b1ec2e7dbf2707ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:42.299117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:42.485394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1152/2291] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:42.489014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:42.494248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:42.544513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce80a8-55ed-415c-9272-41b185e6ce7e service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:42.552382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:42.553996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:42.554912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:42.555296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:42.556906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:42.558996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:42.586465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:42.694426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1140/2292] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:42 2022] POST /identity/v3/projects => generated 371 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:42.700822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:42.714903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38034af-c815-4a22-a0e6-aeb1678e6877 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:42.839171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:42.839171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:42.839171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:42.839171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:42.839957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:42.842821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:42.996607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:42.998414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:43.005296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1153/2293] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:42 2022] POST /identity/v3/users => generated 292 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:43.009766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.038469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a758a3-e32e-41e1-ae38-a494ee71d1a3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.042104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28224f2-9e44-40c9-ba1a-698895e2fd5d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.044659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.045375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.045715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.047980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] PATH_INFO: `/v3/projects/e5e3d5ca106841cea8f4f07cfc54ca21` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.047980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e3d5ca106841cea8f4f07cfc54ca21)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.049936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.058657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.059318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.059604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.059881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.060818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.066109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.114788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1154/2294] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:43.115790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.130548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb5f5f-3d72-49b6-9e00-3b58a41b5a4e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.138401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.139247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.139537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.140409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] PATH_INFO: `/v3/projects/10d1be733d35463181aea92f1fcfa3f8/users/a60bf4f434b74d47b0c73c60177d101c/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.154387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:43.166958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d1be733d35463181aea92f1fcfa3f8, user_id=a60bf4f434b74d47b0c73c60177d101c, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.172617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.205936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e3d5ca106841cea8f4f07cfc54ca21', 'request_id': 'req-f3fa9f32-246d-490f-88d0-7f8c3288c170'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:43.217136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e3d5ca106841cea8f4f07cfc54ca21', 'request_id': 'req-f3fa9f32-246d-490f-88d0-7f8c3288c170'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:43.257167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1155/2295] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:43 2022] PUT /identity/v3/projects/10d1be733d35463181aea92f1fcfa3f8/users/a60bf4f434b74d47b0c73c60177d101c/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:43.258456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.336770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e3d5ca106841cea8f4f07cfc54ca21', 'request_id': 'req-f3fa9f32-246d-490f-88d0-7f8c3288c170'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:43.357889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] The token cache is being invalidate because project e5e3d5ca106841cea8f4f07cfc54ca21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:41:43.359480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:43.361270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f3fa9f32-246d-490f-88d0-7f8c3288c170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:41:43.366012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1141/2296] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:41:42 2022] DELETE /identity/v3/projects/e5e3d5ca106841cea8f4f07cfc54ca21 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:43.372164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.372727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.373740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.383974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729f35b-29e2-486d-b3ea-6671a44c833b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.398963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe07527-33f8-4957-8c85-ffe07bc80263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.401081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe07527-33f8-4957-8c85-ffe07bc80263 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.401394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe07527-33f8-4957-8c85-ffe07bc80263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.401831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe07527-33f8-4957-8c85-ffe07bc80263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.402966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe07527-33f8-4957-8c85-ffe07bc80263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.404189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe07527-33f8-4957-8c85-ffe07bc80263 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.410458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:43.411936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:43.431382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1156/2297] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:43.431603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1142/2298] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:41:43.432445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.432445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.432687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.433086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.434637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:43.456801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723b831-779d-41ed-b3e3-1d4031d4404f None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.467638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.468165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:43.472981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.473274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:43.473402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.473402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] PATH_INFO: `/v3/projects/10d1be733d35463181aea92f1fcfa3f8/users/a60bf4f434b74d47b0c73c60177d101c/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.474663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d1be733d35463181aea92f1fcfa3f8, user_id=a60bf4f434b74d47b0c73c60177d101c, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.477593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.481729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1157/2299] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:43.485833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.493122 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1143/2300] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:43 2022] PUT /identity/v3/projects/10d1be733d35463181aea92f1fcfa3f8/users/a60bf4f434b74d47b0c73c60177d101c/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:43.496135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03499a0c-8c31-4288-8cdf-8ed597de0553 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.597009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.598874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.599237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.599573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.601244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:41:43.601952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.604412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.644420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1158/2301] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:41:43.645548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.662890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411dddd-69cb-44f4-8e9d-1c910767cca7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.674917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.676440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.676775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.678782 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.679891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:43.684665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:43.837242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1144/2302] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:43 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:43.842024 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:43.853184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43df077-04bd-47ed-b96e-cd1d2e9829b4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:43.994750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:43.995691 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:43.996072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:43.997098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:43.997282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:44.006955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:44.132612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1159/2303] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:43 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:44.134936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:44.151955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eb715-1464-4e12-b60b-69bc4331b52e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:44.165136 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:44.166086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:44.168625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:44.169363 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:44.169621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:44.171290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:44.191858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1145/2304] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:44.196283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:44.199316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:44.211185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ac2b8-081d-4b44-9310-dbb6639f880e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:44.217442 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6829e97-2cf7-47c5-9ba2-8a6e0ef1f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:44.218502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6829e97-2cf7-47c5-9ba2-8a6e0ef1f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:44.218884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6829e97-2cf7-47c5-9ba2-8a6e0ef1f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:44.219416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6829e97-2cf7-47c5-9ba2-8a6e0ef1f0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:44.220116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6829e97-2cf7-47c5-9ba2-8a6e0ef1f0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:44.222128 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6829e97-2cf7-47c5-9ba2-8a6e0ef1f0e1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:44.236666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1160/2305] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:44.244707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:44.258670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c525fc-463b-420f-87cc-0888c02b5b17 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:44.273155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:44.273886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:44.274174 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:44.274466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] PATH_INFO: `/v3/projects/32813adc26c64016b99c9c5fd838d391/users/ac48e585c133487dadbe0a1e095bfde0/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:44.296151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32813adc26c64016b99c9c5fd838d391, user_id=ac48e585c133487dadbe0a1e095bfde0, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:44.299984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:44.408278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1146/2306] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:44 2022] PUT /identity/v3/projects/32813adc26c64016b99c9c5fd838d391/users/ac48e585c133487dadbe0a1e095bfde0/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:44.411524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359b5b6-cba8-47c7-ae15-5184f9e60672 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:44.411524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359b5b6-cba8-47c7-ae15-5184f9e60672 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:44.411524 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359b5b6-cba8-47c7-ae15-5184f9e60672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:44.469500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b359b5b6-cba8-47c7-ae15-5184f9e60672 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:44.471821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b359b5b6-cba8-47c7-ae15-5184f9e60672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:44.615217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1161/2307] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:44.629776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:44.632117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:44.745858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd699118-a2a5-481f-b0fb-a44289f4ee92 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:44.762321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:44.763142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:44.764088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:44.764480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:44.780195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:44.782684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:44.799286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1147/2308] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:46.420489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285b1e-3673-4252-8064-efd25dffd712 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:46.420994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285b1e-3673-4252-8064-efd25dffd712 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:46.421099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9285b1e-3673-4252-8064-efd25dffd712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:46.467808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f9285b1e-3673-4252-8064-efd25dffd712 None None] MFA Rules not processed for user `55beb38ea2994ea7987fe546235b58a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:46.469053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f9285b1e-3673-4252-8064-efd25dffd712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:46.621414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1162/2309] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:46.636559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:46.639309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:46.654226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:46.787916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137f5d2-e397-485d-bb98-e3e7ece2d96b service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:46.800710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f419466-9b8d-4891-bedf-5c6eb9a34b17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:46.801674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f419466-9b8d-4891-bedf-5c6eb9a34b17 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:46.802077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f419466-9b8d-4891-bedf-5c6eb9a34b17 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:46.802520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f419466-9b8d-4891-bedf-5c6eb9a34b17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:46.823445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f419466-9b8d-4891-bedf-5c6eb9a34b17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:46.825102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f419466-9b8d-4891-bedf-5c6eb9a34b17 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:46.848617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1148/2310] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:51.054229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ac6bc-31fd-42fa-a4b3-7914192e2d08 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:51.054775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ac6bc-31fd-42fa-a4b3-7914192e2d08 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:51.054873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ac6bc-31fd-42fa-a4b3-7914192e2d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:51.078659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-611ac6bc-31fd-42fa-a4b3-7914192e2d08 None None] MFA Rules not processed for user `55beb38ea2994ea7987fe546235b58a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:51.080304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-611ac6bc-31fd-42fa-a4b3-7914192e2d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:51.107755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1163/2311] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:54.971023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:54.971536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:54.971617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:55.016666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] MFA Rules not processed for user `3ec8d131a9764028a096314a4ecd58f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:55.018224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:55.100399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1149/2312] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:56.823931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91789a5-ee41-4bea-b070-fab2ab443aa5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:56.824387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91789a5-ee41-4bea-b070-fab2ab443aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:56.824480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91789a5-ee41-4bea-b070-fab2ab443aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:56.877250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c91789a5-ee41-4bea-b070-fab2ab443aa5 None None] MFA Rules not processed for user `a60bf4f434b74d47b0c73c60177d101c`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:56.880373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c91789a5-ee41-4bea-b070-fab2ab443aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:57.035725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:57.042852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1164/2313] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:57.062002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:57.065325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:57.104707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27c7ef-b090-4a01-8838-47591f4ab7d0 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:57.130931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:57.130931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:57.130931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:57.130931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:57.136175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:57.232987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:57.234989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:57.255857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1150/2314] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:41:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:41:57.887151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:57.887646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:57.887646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:57.933604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] MFA Rules not processed for user `ac48e585c133487dadbe0a1e095bfde0`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:41:57.935316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:41:58.061950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1165/2315] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:41:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:41:58.063205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:58.080211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb497b-5cf7-4928-8781-ae0ccad0d542 service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:58.090108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:58.091087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:58.091362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:58.095929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:58.097386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:58.100990 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:58.198860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1151/2316] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:41:58 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:41:58.205567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:58.215999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee6d0a-b3a0-429b-ac7f-af93f743ee3e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:58.305702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:58.307802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:58.308351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:58.308855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:58.309812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:58.312401 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:58.396719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1166/2317] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:41:58 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:41:58.401243 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:58.424451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67375b7e-4cf0-4ff6-9a75-e7ee40372290 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:58.434615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:58.435577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:58.436090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:58.436771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:58.437918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:58.440036 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:58.469754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:41:58.487854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1152/2318] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:58.489804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:58.513869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e4b11-d433-487a-83a4-ddf2dee8b741 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:58.530378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:58.531082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:58.531361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:58.531628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] PATH_INFO: `/v3/projects/19fd5873e36042d79e2ee534175828f5/users/73471e443c784730821030edabb8804f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:58.562276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fd5873e36042d79e2ee534175828f5, user_id=73471e443c784730821030edabb8804f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:58.591432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:58.722937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1167/2319] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:58 2022] PUT /identity/v3/projects/19fd5873e36042d79e2ee534175828f5/users/73471e443c784730821030edabb8804f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:41:58.723962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:58.753295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7351f1-afbf-4cc3-bcf5-20b4e9c07637 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:58.871491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005803e-8e4d-4562-bf51-2f41102d4d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:58.872727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005803e-8e4d-4562-bf51-2f41102d4d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:58.873329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005803e-8e4d-4562-bf51-2f41102d4d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:58.873960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005803e-8e4d-4562-bf51-2f41102d4d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:58.875352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005803e-8e4d-4562-bf51-2f41102d4d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:58.879299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005803e-8e4d-4562-bf51-2f41102d4d52 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:58.901614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1153/2320] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:41:58.902046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:41:58.919416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6227c-156b-4186-9175-3b25c5d0511a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:41:58.929381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd15ff-be33-4d25-82d7-b6b3566fa9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:41:58.933555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd15ff-be33-4d25-82d7-b6b3566fa9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:41:58.933555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd15ff-be33-4d25-82d7-b6b3566fa9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:41:58.933555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd15ff-be33-4d25-82d7-b6b3566fa9f4 admin admin] PATH_INFO: `/v3/projects/19fd5873e36042d79e2ee534175828f5/users/73471e443c784730821030edabb8804f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:41:58.951530 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd15ff-be33-4d25-82d7-b6b3566fa9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fd5873e36042d79e2ee534175828f5, user_id=73471e443c784730821030edabb8804f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:41:58.966661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd15ff-be33-4d25-82d7-b6b3566fa9f4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:41:59.004975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1168/2321] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:41:58 2022] PUT /identity/v3/projects/19fd5873e36042d79e2ee534175828f5/users/73471e443c784730821030edabb8804f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:42:00.961433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:00.963017 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:00.963525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:00.997324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] MFA Rules not processed for user `dcc12077b11448a3b1ec2e7dbf2707ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:00.998964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:01.096986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1154/2322] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:05.084087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889b7cd-6f24-49e9-a7f9-b407d943718d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:05.085358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889b7cd-6f24-49e9-a7f9-b407d943718d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:05.086069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889b7cd-6f24-49e9-a7f9-b407d943718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:05.205880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0889b7cd-6f24-49e9-a7f9-b407d943718d None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:05.211298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0889b7cd-6f24-49e9-a7f9-b407d943718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:05.283548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:05.346576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1169/2323] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:42:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:42:05.408692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:05.410258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:05.622356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb9473-e203-4b31-8923-589cb912cb75 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:05.628690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:05.629521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:05.629856 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:05.630218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:05.640153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:05.646238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:05.652675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1155/2324] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:42:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:42:07.661981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:07.662545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:07.662545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:07.692508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] MFA Rules not processed for user `73471e443c784730821030edabb8804f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:07.693803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:07.787562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1170/2325] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:07.804627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:07.840107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:07.887073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bcdfd-97df-4532-ae3d-737d06a42230 service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:07.893906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11791b9-2bf4-45b1-ac0b-bf8767b4c074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:07.894596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11791b9-2bf4-45b1-ac0b-bf8767b4c074 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:07.894884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11791b9-2bf4-45b1-ac0b-bf8767b4c074 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:07.895165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11791b9-2bf4-45b1-ac0b-bf8767b4c074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:07.968309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11791b9-2bf4-45b1-ac0b-bf8767b4c074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:07.970916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11791b9-2bf4-45b1-ac0b-bf8767b4c074 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:07.986804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1156/2326] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:42:08.158263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:08.161275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:08.286996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f75d12-11a6-4db3-ae81-111d50051501 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:08.293358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:08.294266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:08.294705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:08.295131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] PATH_INFO: `/v3/users/cbedb8d08a5f4122b83b12dc29a175f5` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:08.296685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbedb8d08a5f4122b83b12dc29a175f5)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:08.300525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:08.474930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbedb8d08a5f4122b83b12dc29a175f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:08.511833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbedb8d08a5f4122b83b12dc29a175f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:08.511833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbedb8d08a5f4122b83b12dc29a175f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:08.521514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:08.523623 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbedb8d08a5f4122b83b12dc29a175f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:08.577866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbedb8d08a5f4122b83b12dc29a175f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:08.592246 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1171/2327] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:42:08 2022] DELETE /identity/v3/users/cbedb8d08a5f4122b83b12dc29a175f5 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:08.594802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:08.594954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:08.595189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:08.623685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:08.625275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:08.637306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:08.638796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:08.726676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323ef40-e61d-4a19-a4bd-a97913532a99 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:08.734634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:08.736187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:08.736736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:08.737298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] PATH_INFO: `/v3/users/5bd06a639d6e41a3ae3b9d5eb9b7ce79` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:08.739731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd06a639d6e41a3ae3b9d5eb9b7ce79)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:08.742114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:08.765360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1157/2328] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:08.776417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:08.778572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:08.863182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5bbd6-e27f-471a-baa1-1f380f96e244 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:08.869821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88eac5-14dc-4b6e-9490-3e34ccd4cfe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:08.876402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88eac5-14dc-4b6e-9490-3e34ccd4cfe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:08.876885 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88eac5-14dc-4b6e-9490-3e34ccd4cfe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:08.877175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88eac5-14dc-4b6e-9490-3e34ccd4cfe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:08.889804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88eac5-14dc-4b6e-9490-3e34ccd4cfe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:08.891419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88eac5-14dc-4b6e-9490-3e34ccd4cfe0 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:08.906257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1158/2329] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:42:09.056978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd06a639d6e41a3ae3b9d5eb9b7ce79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.086317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd06a639d6e41a3ae3b9d5eb9b7ce79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.095969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd06a639d6e41a3ae3b9d5eb9b7ce79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.124617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd06a639d6e41a3ae3b9d5eb9b7ce79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.226198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd06a639d6e41a3ae3b9d5eb9b7ce79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.234151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1172/2330] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:42:08 2022] DELETE /identity/v3/users/5bd06a639d6e41a3ae3b9d5eb9b7ce79 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:42:09.239005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:09.239331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:09.239567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:09.267266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:09.269405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:09.279273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:09.420144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1159/2331] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:09.430226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:09.431520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:09.563077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:09.564634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:09.616368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84113-8620-4b8e-ab56-da216aa7fd23 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:09.620536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36b664-e143-488f-b93e-4178902b9c3e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:09.627459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:09.629464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:09.629872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:09.630458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:09.631217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:09.631521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:09.631973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:09.632411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] PATH_INFO: `/v3/projects/2f31c28caabb4a1cbe5d0d5320ca0ad8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:09.636079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f31c28caabb4a1cbe5d0d5320ca0ad8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:09.638554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:09.662757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:09.665794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:09.694629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1173/2332] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:42:09.820741 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f31c28caabb4a1cbe5d0d5320ca0ad8', 'request_id': 'req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.837498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f31c28caabb4a1cbe5d0d5320ca0ad8', 'request_id': 'req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.913751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f31c28caabb4a1cbe5d0d5320ca0ad8', 'request_id': 'req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.941238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] The token cache is being invalidate because project 2f31c28caabb4a1cbe5d0d5320ca0ad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:42:09.941529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.942609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c5628f89-9d59-4217-a5e5-74b5c9bad6f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:09.944573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1160/2333] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:42:09 2022] DELETE /identity/v3/projects/2f31c28caabb4a1cbe5d0d5320ca0ad8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:42:10.518792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:10.522650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:10.566844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b991f-d667-4711-81d6-4182c0402929 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:10.605900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:10.824709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:10.825357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:10.825646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:10.830287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] PATH_INFO: `/v3/projects/415a1461fd1344acb8828e0d78d03717` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:10.834874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=415a1461fd1344acb8828e0d78d03717)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:10.848917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:11.078959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415a1461fd1344acb8828e0d78d03717', 'request_id': 'req-7da05056-e5de-4f39-b089-cf08222dc61e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:11.090775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415a1461fd1344acb8828e0d78d03717', 'request_id': 'req-7da05056-e5de-4f39-b089-cf08222dc61e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:11.183166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415a1461fd1344acb8828e0d78d03717', 'request_id': 'req-7da05056-e5de-4f39-b089-cf08222dc61e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:11.206943 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] The token cache is being invalidate because project 415a1461fd1344acb8828e0d78d03717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:42:11.207692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:11.208734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7da05056-e5de-4f39-b089-cf08222dc61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:11.216058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1174/2334] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:42:10 2022] DELETE /identity/v3/projects/415a1461fd1344acb8828e0d78d03717 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:11.216850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:11.217317 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:11.221638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:11.247219 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:11.248912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:11.344404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1161/2335] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:42:11.351480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:11.353391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:11.353635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:11.380759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:11.385346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:11.397746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1175/2336] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:11.402040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:11.416216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1610123-ec83-4067-8c0d-7533dbfd2341 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:11.422483 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:11.423694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:11.424195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:11.424942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:11.426002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:42:11.426526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:11.428550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:11.453522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1162/2337] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:42:11.458161 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:11.496756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25cea4-7392-4cbd-9645-59a58a8dc443 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:11.515069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:11.516040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:11.516961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:11.517600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:11.518560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:11.521795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:11.673896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1176/2338] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:42:11 2022] POST /identity/v3/projects => generated 369 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:42:11.675961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:11.696971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f8f89-63ef-45df-9d83-2ef4fffc327f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:11.733389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:11.862988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:11.864265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:11.864814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:11.865302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:11.866282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:11.868645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:12.108281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1163/2339] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:42:11 2022] POST /identity/v3/users => generated 285 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:42:12.108777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:12.125558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc6a92-5371-4521-9bd7-0ce0cc071272 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:12.132074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:12.132958 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:12.133267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:12.134325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:12.135881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:12.137568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:12.148705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1177/2340] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:42:12.153222 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:12.173333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e75ebe-d803-468c-a061-cbbbec2c2c17 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:12.181697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7806e9-c6f0-4714-9493-05763c932cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:12.182547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7806e9-c6f0-4714-9493-05763c932cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:12.183113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7806e9-c6f0-4714-9493-05763c932cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:12.183426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7806e9-c6f0-4714-9493-05763c932cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:12.184489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7806e9-c6f0-4714-9493-05763c932cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:12.187153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7806e9-c6f0-4714-9493-05763c932cb5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:12.242992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1164/2341] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:42:12.247346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:12.263229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88da9b-f150-46fb-ae3b-4d8c25bcb4a8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:12.273764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:12.274409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:12.275275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:12.275570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] PATH_INFO: `/v3/projects/75c51ae2d88a462680438362e0bedb87/users/39795773720f45fc987000e1fa704278/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:12.335151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c51ae2d88a462680438362e0bedb87, user_id=39795773720f45fc987000e1fa704278, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:12.340056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:12.480508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1178/2342] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:42:12 2022] PUT /identity/v3/projects/75c51ae2d88a462680438362e0bedb87/users/39795773720f45fc987000e1fa704278/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:12.492055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:12.492438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:12.492890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:12.563004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] MFA Rules not processed for user `39795773720f45fc987000e1fa704278`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:12.564878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:12.776386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1165/2343] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:20.220487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:20.222759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:20.299005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:20.331805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df5101-67a9-4b99-a658-75b05c5388d2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:20.336011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecce56-a627-42c1-9892-3b43624723d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:20.337184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecce56-a627-42c1-9892-3b43624723d9 service glance] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:20.337493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecce56-a627-42c1-9892-3b43624723d9 service glance] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:20.337763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecce56-a627-42c1-9892-3b43624723d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:20.347283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecce56-a627-42c1-9892-3b43624723d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:20.351171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecce56-a627-42c1-9892-3b43624723d9 service glance] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:20.359827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1179/2344] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:42:22.853673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:30.632951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=9 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:35.795676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:35.797074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:35.870287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b107b3-d0c8-408c-943e-380bfeef7f6c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:35.876533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:35.877168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:35.877480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:35.877753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:35.879643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:35.892883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:35.894498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:35.905088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1166/2345] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:42:39.185568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:39.185971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:39.186274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:39.206468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:39.207796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:39.267048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1180/2346] 10.222.0.66 () {58 vars in 967 bytes} [Thu Aug 4 03:42:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:39.280551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:39.282094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:39.356770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6d63e-43b4-4bdf-9da7-80fb6c075bf8 service placement] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:39.361962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:39.362582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:39.362865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:39.363148 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:39.372263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:39.374056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:39.382579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1167/2347] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:42:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:42:42.331073 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:42.332492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:42.400807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fdc3d0-9aff-4b55-8062-27543e83f48f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:42.405649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:42.406437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:42.406589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:42.406900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] PATH_INFO: `/v3/users/d8a3eed3ee0b42f69948aced53729d6f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:42.408164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a3eed3ee0b42f69948aced53729d6f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:42.413483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:42.583698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a3eed3ee0b42f69948aced53729d6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:42.592340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a3eed3ee0b42f69948aced53729d6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:42.593345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a3eed3ee0b42f69948aced53729d6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:42.607097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a3eed3ee0b42f69948aced53729d6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:42.684027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5a44be42-7d0a-4642-be7c-ba4045f0c139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a3eed3ee0b42f69948aced53729d6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:42.690373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1181/2348] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:42:42 2022] DELETE /identity/v3/users/d8a3eed3ee0b42f69948aced53729d6f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:42.926667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:42.941324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad05cf-93c1-4a54-9dee-6e2fb5aab161 service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.006460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.007103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.007425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.008966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] PATH_INFO: `/v3/projects/1c4daab5960f4c98874faea2066b1909` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.008966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4daab5960f4c98874faea2066b1909)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.011507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.039442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:43.102897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4daab5960f4c98874faea2066b1909', 'request_id': 'req-2a741f3e-ca9d-4088-baa1-763e3e30f74d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:43.110487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4daab5960f4c98874faea2066b1909', 'request_id': 'req-2a741f3e-ca9d-4088-baa1-763e3e30f74d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:43.208992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4daab5960f4c98874faea2066b1909', 'request_id': 'req-2a741f3e-ca9d-4088-baa1-763e3e30f74d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:43.220819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] The token cache is being invalidate because project 1c4daab5960f4c98874faea2066b1909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:42:43.221110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:43.221774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2a741f3e-ca9d-4088-baa1-763e3e30f74d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:42:43.223296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1168/2349] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:42:42 2022] DELETE /identity/v3/projects/1c4daab5960f4c98874faea2066b1909 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:43.229146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.229395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.229696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.250335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:43.251550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:43.269622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:43.311703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1182/2350] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:43.319682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56959189-153f-4394-b14a-a10f4d597601 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.319957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56959189-153f-4394-b14a-a10f4d597601 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.320242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56959189-153f-4394-b14a-a10f4d597601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.340759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-56959189-153f-4394-b14a-a10f4d597601 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:43.342003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-56959189-153f-4394-b14a-a10f4d597601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:43.349901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1169/2351] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:43.355382 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:43.366933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8a837-2945-466f-ad43-08c8fbdd6c6f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.371770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.372395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.372678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.373005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.373701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:42:43.374008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.375634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.387330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1183/2352] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:42:43.391649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56959189-153f-4394-b14a-a10f4d597601 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:43.401416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56959189-153f-4394-b14a-a10f4d597601 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.406783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.407470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.407759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.408123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.408899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.411151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.560635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1170/2353] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:42:43 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:42:43.564873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:43.576303 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b032ec3-9433-459e-9bce-23a038e6b725 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.633828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.634451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.635352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.635629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.636412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.638393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.741947 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1184/2354] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:42:43 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:42:43.746874 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:43.758511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4476245-7e2f-433f-89cf-dac233c87522 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.765074 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.765727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.766006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.766280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.768607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.768839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.781009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1171/2355] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:42:43.784825 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:43.795853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6a277-c618-438a-b0a5-397fdd83dd0b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.800528 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ef12c-57a4-4cd7-be27-8c5dbbf114b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.801150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ef12c-57a4-4cd7-be27-8c5dbbf114b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.801469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ef12c-57a4-4cd7-be27-8c5dbbf114b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.801750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ef12c-57a4-4cd7-be27-8c5dbbf114b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.802486 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ef12c-57a4-4cd7-be27-8c5dbbf114b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.804102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ef12c-57a4-4cd7-be27-8c5dbbf114b3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.815666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1185/2356] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:42:43.819974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:43.832831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae4c07-0a83-422d-aeaf-880ae56105fe admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:43.837720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:43.839001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.839276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.839552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] PATH_INFO: `/v3/projects/2049d7705d5c4afe94c22a6f1284327b/users/d245c731eb58483bb635a7e70542b892/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.855201 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2049d7705d5c4afe94c22a6f1284327b, user_id=d245c731eb58483bb635a7e70542b892, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:43.857920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:43.925069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1172/2357] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:42:43 2022] PUT /identity/v3/projects/2049d7705d5c4afe94c22a6f1284327b/users/d245c731eb58483bb635a7e70542b892/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:43.930621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b6940-7aa8-4091-b050-f8a19d37018f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:43.930917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b6940-7aa8-4091-b050-f8a19d37018f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:43.931212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b6940-7aa8-4091-b050-f8a19d37018f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:43.951185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-1d3b6940-7aa8-4091-b050-f8a19d37018f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:43.952406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1d3b6940-7aa8-4091-b050-f8a19d37018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:44.017665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1186/2358] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:44.028664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:44.030484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:44.090298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652ae91-034f-4578-ae68-6b0b9daa9248 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:44.093904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:44.094510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:44.094788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:44.095057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:44.105502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:44.106966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:44.117071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1173/2359] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:42:50.125829 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05400073-c070-49b1-b371-26e3553961f1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.126296 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05400073-c070-49b1-b371-26e3553961f1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.126498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05400073-c070-49b1-b371-26e3553961f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.159622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-05400073-c070-49b1-b371-26e3553961f1 None None] MFA Rules not processed for user `d245c731eb58483bb635a7e70542b892`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:50.165747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-05400073-c070-49b1-b371-26e3553961f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:50.195402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:50.254214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1187/2360] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:50.258350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:50.269320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bc9c5-b003-41a8-89f0-83315e6dce80 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:50.274306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:50.274929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.275434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.275709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.276587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:50.278579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:50.390054 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1174/2361] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:42:50 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:42:50.394211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05400073-c070-49b1-b371-26e3553961f1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:50.407395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05400073-c070-49b1-b371-26e3553961f1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:50.475084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:50.475084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.475084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.475581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.488558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:50.488558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:50.571588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1188/2362] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:42:50 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:42:50.576354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:50.588114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecb83d-40a5-46dd-b69a-e52db452be7e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:50.597415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:50.598301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.599164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.599626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.601968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:50.605055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:50.623767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1175/2363] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:42:50.626859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:50.638231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9c177-5303-445c-9ed6-fd43c2126a33 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:50.643322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:50.644103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.644389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.644662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] PATH_INFO: `/v3/projects/a0164049927c4e658615cd0cba7a4774/users/cf10316294aa44919aae74d21819af3c/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.672724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0164049927c4e658615cd0cba7a4774, user_id=cf10316294aa44919aae74d21819af3c, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:50.675677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:50.795032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1189/2364] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:42:50 2022] PUT /identity/v3/projects/a0164049927c4e658615cd0cba7a4774/users/cf10316294aa44919aae74d21819af3c/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:42:50.798980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:50.812532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dd195-017c-447f-b87e-d2620b6a819b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:50.827552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:50.884242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:50.884919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.885237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.885690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.886393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:50.888029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:50.898683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1176/2365] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:42:50.903889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:50.918587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b292827-f1bd-4097-bd13-ca6f9a6d83d8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:50.923774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a6a86-6e5b-414d-95b3-5d0e176c34c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:50.924492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6a86-6e5b-414d-95b3-5d0e176c34c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:50.925113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6a86-6e5b-414d-95b3-5d0e176c34c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:50.925113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a6a86-6e5b-414d-95b3-5d0e176c34c9 admin admin] PATH_INFO: `/v3/projects/a0164049927c4e658615cd0cba7a4774/users/cf10316294aa44919aae74d21819af3c/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:50.927247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6a86-6e5b-414d-95b3-5d0e176c34c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0164049927c4e658615cd0cba7a4774, user_id=cf10316294aa44919aae74d21819af3c, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:50.930951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a6a86-6e5b-414d-95b3-5d0e176c34c9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:50.945439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1190/2366] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:42:50 2022] PUT /identity/v3/projects/a0164049927c4e658615cd0cba7a4774/users/cf10316294aa44919aae74d21819af3c/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:42:55.356986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:55.419908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac84b63-9c27-4498-b376-6a015e08ead5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:55.477673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:55.479099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:55.479412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:55.479665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:55.481738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:55.555965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:55.559707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:55.577664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1177/2367] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:42:56.667841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ff782-3918-476f-9cc4-1efdfa015336 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:56.668684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ff782-3918-476f-9cc4-1efdfa015336 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:56.669114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ff782-3918-476f-9cc4-1efdfa015336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:56.709071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-993ff782-3918-476f-9cc4-1efdfa015336 None None] MFA Rules not processed for user `cf10316294aa44919aae74d21819af3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:56.710365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-993ff782-3918-476f-9cc4-1efdfa015336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:56.797981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:56.810692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1191/2368] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:42:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:56.824587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:56.826930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:56.855947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:42:56.966281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1d74c-7299-41a2-8e16-588034b0ccc6 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:56.971656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:56.972896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:56.973652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:56.974078 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:57.088698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:57.091239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:57.114964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1178/2369] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:42:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:42:57.389529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f476cb-5be3-4fb3-8e45-525bccd8c2cb None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:57.389871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f476cb-5be3-4fb3-8e45-525bccd8c2cb None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:57.390094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f476cb-5be3-4fb3-8e45-525bccd8c2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:57.414416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-43f476cb-5be3-4fb3-8e45-525bccd8c2cb None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:42:57.416576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-43f476cb-5be3-4fb3-8e45-525bccd8c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:57.489928 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1192/2370] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:42:57.503775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:42:57.505332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:42:57.611436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce587b7-115f-41c1-aa8a-d61cf38549cb service glance] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:42:57.617816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533bfb2-fe38-4fb1-8466-91f2f592b1eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:42:57.618552 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533bfb2-fe38-4fb1-8466-91f2f592b1eb service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:42:57.619451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533bfb2-fe38-4fb1-8466-91f2f592b1eb service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:42:57.619786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533bfb2-fe38-4fb1-8466-91f2f592b1eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:42:57.629260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533bfb2-fe38-4fb1-8466-91f2f592b1eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:42:57.630989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533bfb2-fe38-4fb1-8466-91f2f592b1eb service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:42:57.642162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1179/2371] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:42:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:43:07.480703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=26 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:07.637571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=72 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:08.374510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26fa2-ee51-412b-8ded-f8d9b5d12ccc None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:08.374798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26fa2-ee51-412b-8ded-f8d9b5d12ccc None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:08.375092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e26fa2-ee51-412b-8ded-f8d9b5d12ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:08.394386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-78e26fa2-ee51-412b-8ded-f8d9b5d12ccc None None] MFA Rules not processed for user `d245c731eb58483bb635a7e70542b892`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:08.395594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-78e26fa2-ee51-412b-8ded-f8d9b5d12ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:08.402579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1193/2372] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:16.308262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:16.308828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:16.308947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:16.341927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] MFA Rules not processed for user `ac48e585c133487dadbe0a1e095bfde0`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:16.344724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:16.408255 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1180/2373] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:17.071663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:17.072144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:17.072254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:17.097437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:17.098763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:17.107917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1194/2374] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:43:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:17.120944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:17.135387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6ab91-8d5a-42d8-a44c-d5d4ff62e862 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:17.141681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532d14c-ce1e-4022-ba32-4dd249d1dca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:17.142422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d14c-ce1e-4022-ba32-4dd249d1dca3 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:17.142776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d14c-ce1e-4022-ba32-4dd249d1dca3 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:17.143117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d14c-ce1e-4022-ba32-4dd249d1dca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:17.175807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532d14c-ce1e-4022-ba32-4dd249d1dca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:17.178104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532d14c-ce1e-4022-ba32-4dd249d1dca3 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:17.195166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1181/2375] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:43:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 03:43:20.819403 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:20.821869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:20.909820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e1f3c-f438-4dd4-b5ff-59154b556cf6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:20.914766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:20.915472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:20.915693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:20.915964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] PATH_INFO: `/v3/users/39795773720f45fc987000e1fa704278` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:20.917885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=39795773720f45fc987000e1fa704278)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:20.919735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:21.061998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39795773720f45fc987000e1fa704278'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:21.071352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39795773720f45fc987000e1fa704278'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:21.072139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39795773720f45fc987000e1fa704278'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:21.089038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39795773720f45fc987000e1fa704278'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:21.146692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39795773720f45fc987000e1fa704278'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:21.161759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1195/2376] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:43:20 2022] DELETE /identity/v3/users/39795773720f45fc987000e1fa704278 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:43:21.161918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:21.162356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:21.162356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:21.182041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:21.183313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:21.249189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1182/2377] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:21.258817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:21.260749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:21.326580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68721e-a9c0-4988-9e55-6008826da16d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:21.330287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cbad2b-c9ce-4de0-8133-fb0be9ff22dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:21.330904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbad2b-c9ce-4de0-8133-fb0be9ff22dd service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:21.331184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbad2b-c9ce-4de0-8133-fb0be9ff22dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:21.331457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbad2b-c9ce-4de0-8133-fb0be9ff22dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:21.342231 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbad2b-c9ce-4de0-8133-fb0be9ff22dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:21.344119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cbad2b-c9ce-4de0-8133-fb0be9ff22dd service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:21.356585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1196/2378] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:43:21.778960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:21.780392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:21.793190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc1831-448d-4b54-87e0-9cf211199d87 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:21.798898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:21.800260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:21.800260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:21.800598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] PATH_INFO: `/v3/projects/75c51ae2d88a462680438362e0bedb87` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:21.801948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c51ae2d88a462680438362e0bedb87)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:21.805111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:21.918455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c51ae2d88a462680438362e0bedb87', 'request_id': 'req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:21.924742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c51ae2d88a462680438362e0bedb87', 'request_id': 'req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:22.091980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c51ae2d88a462680438362e0bedb87', 'request_id': 'req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:22.096110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] The token cache is being invalidate because project 75c51ae2d88a462680438362e0bedb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:43:22.096407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:22.097082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c69d81af-79d8-4ba1-9d7a-c45d0df801ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:22.098644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1183/2379] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:43:21 2022] DELETE /identity/v3/projects/75c51ae2d88a462680438362e0bedb87 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:43:22.104421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:22.105468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:22.105468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:22.129085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:22.130738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:22.135491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:22.235334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1197/2380] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:22.242512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:22.243085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:22.243085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:22.269774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:22.271185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:22.279336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1184/2381] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:22.283934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:22.297549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7eceeb-934c-4bf4-9e5e-5271c0e311c7 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:22.302809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:22.303455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:22.303735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:22.304006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:22.304718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:43:22.305020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:22.306684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:22.321181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1198/2382] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:43:22.325852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:22.336945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782a1ca-edb1-4849-a5d3-f68cf9fa48af None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:22.341989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:22.342749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:22.343033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:22.343307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:22.343994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:22.346023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:22.351712 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:22.782309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1185/2383] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:43:22 2022] POST /identity/v3/projects => generated 373 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:43:22.786760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:22.797470 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d740efe-f28d-4b87-b346-6c7f64d1e5f7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:22.878315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:22.918935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:22.918935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:22.918935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:22.918935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:22.918935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:23.083742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1199/2384] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:43:22 2022] POST /identity/v3/users => generated 287 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:43:23.087960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:23.098763 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498b63a-ca13-4205-90a1-341bc503f7cd admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:23.103458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:23.104197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.104522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.104920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.105702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:23.107253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:23.118694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1186/2385] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:43:23.123226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:23.134258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825b0a5-0736-440b-8fe9-45a46b6f0cb2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:23.140039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07236fd7-693f-452f-a0eb-92042544af1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:23.148772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07236fd7-693f-452f-a0eb-92042544af1c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.148772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07236fd7-693f-452f-a0eb-92042544af1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.148772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07236fd7-693f-452f-a0eb-92042544af1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.148772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07236fd7-693f-452f-a0eb-92042544af1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:23.148772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07236fd7-693f-452f-a0eb-92042544af1c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:23.158656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1200/2386] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:43:23.159868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:23.176037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c44bea-6486-4745-8615-49ba81fbe17d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:23.180701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:23.181409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.181694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.182546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] PATH_INFO: `/v3/projects/83df4e0c789145d49192a30e4699ff08/users/f987e491f4b74a8183f35270d996ca34/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.197527 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83df4e0c789145d49192a30e4699ff08, user_id=f987e491f4b74a8183f35270d996ca34, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:23.201234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:23.306827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1187/2387] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:43:23 2022] PUT /identity/v3/projects/83df4e0c789145d49192a30e4699ff08/users/f987e491f4b74a8183f35270d996ca34/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:43:23.318329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.318686 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.319026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.347812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] MFA Rules not processed for user `f987e491f4b74a8183f35270d996ca34`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:23.349168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:23.430354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1201/2388] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:23.442024 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:23.443984 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:23.509767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe05a292-99fa-44bf-a848-d470e5b35ad1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:23.514018 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004712e6-bdb8-44d0-b30c-4cde3e6b65a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:23.514684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004712e6-bdb8-44d0-b30c-4cde3e6b65a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.514964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004712e6-bdb8-44d0-b30c-4cde3e6b65a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.515236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004712e6-bdb8-44d0-b30c-4cde3e6b65a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.524749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004712e6-bdb8-44d0-b30c-4cde3e6b65a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:23.526849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004712e6-bdb8-44d0-b30c-4cde3e6b65a0 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:23.535817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1188/2389] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:43:23.560450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:23.566568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:23.631131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf254c1-0e2f-4d25-ae40-265e985e7053 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:23.636275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:23.636900 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.637183 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.637511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] PATH_INFO: `/v3/users/f987e491f4b74a8183f35270d996ca34` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.638683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f987e491f4b74a8183f35270d996ca34)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:23.640614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:23.788514 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f987e491f4b74a8183f35270d996ca34'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:23.795735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f987e491f4b74a8183f35270d996ca34'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:23.796425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f987e491f4b74a8183f35270d996ca34'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:23.809451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f987e491f4b74a8183f35270d996ca34'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:23.871919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f987e491f4b74a8183f35270d996ca34'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:23.879542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1202/2390] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:43:23 2022] DELETE /identity/v3/users/f987e491f4b74a8183f35270d996ca34 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:43:23.884177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:23.884467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:23.884766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:23.905978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:23.907228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:23.965514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1189/2391] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:23.975131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:23.976567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:24.061763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4df3d6-029e-4116-9dac-264d224070ff admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:24.065275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03654b03-1095-4d38-95ae-55a8dbfa0a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:24.065903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03654b03-1095-4d38-95ae-55a8dbfa0a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:24.066177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03654b03-1095-4d38-95ae-55a8dbfa0a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:24.066448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03654b03-1095-4d38-95ae-55a8dbfa0a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:24.076793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03654b03-1095-4d38-95ae-55a8dbfa0a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:24.078305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03654b03-1095-4d38-95ae-55a8dbfa0a25 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:24.088344 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1203/2392] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:43:24.582468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:24.584026 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:24.596731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a347f2e-5d90-412b-b21b-079818c629b5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:24.601935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:24.602673 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:24.603046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:24.603321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] PATH_INFO: `/v3/projects/83df4e0c789145d49192a30e4699ff08` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:24.605032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=83df4e0c789145d49192a30e4699ff08)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:24.607103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:24.714515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83df4e0c789145d49192a30e4699ff08', 'request_id': 'req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:24.715587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:24.721223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83df4e0c789145d49192a30e4699ff08', 'request_id': 'req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:24.791149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83df4e0c789145d49192a30e4699ff08', 'request_id': 'req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:24.802868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] The token cache is being invalidate because project 83df4e0c789145d49192a30e4699ff08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:43:24.803271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:24.803991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d5ec636-b28a-4d89-a4ab-754c4e29ffeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:24.805955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1190/2393] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:43:24 2022] DELETE /identity/v3/projects/83df4e0c789145d49192a30e4699ff08 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:43:24.810980 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:24.811512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:24.811512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:24.831378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:24.832591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:24.854363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:24.896922 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1204/2394] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:43:24.904398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:24.904682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:24.904953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:24.925259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:24.926497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:24.935028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1191/2395] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:24.939335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:24.959049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d95f78-c3de-465a-8dbb-f0c10fd4d5e1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:24.964719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:24.965534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:24.965821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:24.966228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:24.967112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:43:24.967555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:24.969406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:24.984275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1205/2396] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:43:24.988880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:24.999106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f66411-35e3-4492-b6d3-696f23ede730 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:25.004745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:25.005502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:25.005850 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:25.006615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:25.007152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:25.009418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:25.130546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1192/2397] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:43:24 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:43:25.134542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:25.154752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eae1e3-6c34-43b8-ac1e-f9bb79be814c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:25.218930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:25.219835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:25.219967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:25.220147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:25.220971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:25.223392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:25.643933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1206/2398] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:43:25 2022] POST /identity/v3/users => generated 286 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:43:25.654416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:25.664649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabbdf2-7851-4512-8fa0-2f7662648681 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:25.669501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:25.670104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:25.670390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:25.670672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:25.671361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:25.672976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:25.684365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1193/2399] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:43:25.688149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:25.699618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b609c-a421-4c62-a95e-e51bf476229d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:25.704676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1ef6c-0009-4bd8-b3d9-8977096e5f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:25.707783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1ef6c-0009-4bd8-b3d9-8977096e5f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:25.707783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1ef6c-0009-4bd8-b3d9-8977096e5f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:25.707783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1ef6c-0009-4bd8-b3d9-8977096e5f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:25.707783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1ef6c-0009-4bd8-b3d9-8977096e5f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:25.708564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1ef6c-0009-4bd8-b3d9-8977096e5f52 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:25.725950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1207/2400] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:43:25.727119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:25.738238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e439c-a6b2-42eb-86fb-436d62bf15a3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:25.742891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:25.743556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:25.743843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:25.744121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] PATH_INFO: `/v3/projects/a60e2be55ee944f0b59958a831d5a6b0/users/0367481f069e4d3dae7e721cb4888f19/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:25.759639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60e2be55ee944f0b59958a831d5a6b0, user_id=0367481f069e4d3dae7e721cb4888f19, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:25.762982 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:25.833282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1194/2401] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:43:25 2022] PUT /identity/v3/projects/a60e2be55ee944f0b59958a831d5a6b0/users/0367481f069e4d3dae7e721cb4888f19/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:43:25.844324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:25.844893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:25.844893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:25.873266 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] MFA Rules not processed for user `0367481f069e4d3dae7e721cb4888f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:25.875294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:25.945415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1208/2402] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:25.956633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:25.958390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:26.027316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c9431-b9f7-4e6b-b848-1598cd2821a5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:26.047738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd7e03-e905-4134-a188-74a4c67c2df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:26.048386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd7e03-e905-4134-a188-74a4c67c2df7 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:26.048672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd7e03-e905-4134-a188-74a4c67c2df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:26.048949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd7e03-e905-4134-a188-74a4c67c2df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:26.059199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd7e03-e905-4134-a188-74a4c67c2df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:26.060658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd7e03-e905-4134-a188-74a4c67c2df7 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:26.070555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1195/2403] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:43:35.943034 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:36.065868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=59,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:41.239261 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:41.313306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c688791-055b-4b2b-ab5e-1c978e4f10df None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:41.319359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:41.319359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:41.319359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:41.319359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] PATH_INFO: `/v3/users/0367481f069e4d3dae7e721cb4888f19` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:41.329318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0367481f069e4d3dae7e721cb4888f19)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:41.331690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:41.487715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0367481f069e4d3dae7e721cb4888f19'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:41.495283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0367481f069e4d3dae7e721cb4888f19'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:41.496633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0367481f069e4d3dae7e721cb4888f19'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:41.509931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0367481f069e4d3dae7e721cb4888f19'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:41.581181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0367481f069e4d3dae7e721cb4888f19'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:41.587731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1209/2404] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:43:41 2022] DELETE /identity/v3/users/0367481f069e4d3dae7e721cb4888f19 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:43:41.592320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:41.592800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:41.592896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:41.612983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:43:41.614245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:41.677889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1196/2405] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:43:41.687703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:41.689438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:41.745901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e953-1097-43f0-adc3-c5367c06a773 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:41.749618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:41.750227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:41.750510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:41.750784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:41.759492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:41.760912 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:41.770243 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1210/2406] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:43:42.162561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:42.164088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:42.176554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeba55-ccfc-4af1-add2-cb309b9b48d7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:42.182431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:42.183089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:42.184120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:42.184404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] PATH_INFO: `/v3/projects/a60e2be55ee944f0b59958a831d5a6b0` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:42.185641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60e2be55ee944f0b59958a831d5a6b0)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:42.188209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:42.273636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60e2be55ee944f0b59958a831d5a6b0', 'request_id': 'req-3dabe9e9-8c4f-4dd2-915e-78da56f68311'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:42.281422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60e2be55ee944f0b59958a831d5a6b0', 'request_id': 'req-3dabe9e9-8c4f-4dd2-915e-78da56f68311'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:42.374700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60e2be55ee944f0b59958a831d5a6b0', 'request_id': 'req-3dabe9e9-8c4f-4dd2-915e-78da56f68311'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:42.388485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] The token cache is being invalidate because project a60e2be55ee944f0b59958a831d5a6b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:43:42.393086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:42.393086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:43:42.393086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1197/2407] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:43:42 2022] DELETE /identity/v3/projects/a60e2be55ee944f0b59958a831d5a6b0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:43:51.766199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:52.383890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:43:54.820275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:54.821724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:54.833471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afe226-e6e5-4623-b991-e492feaaeed4 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:54.881539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:54.882151 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:54.882535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:54.882813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:54.884581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:54.946586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:54.948035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:54.959039 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1211/2408] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:43:57.454093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:43:57.455623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:57.470754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabe9e9-8c4f-4dd2-915e-78da56f68311 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:43:57.474866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:43:57.475561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:43:57.475913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:43:57.476267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:43:57.478146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:43:57.554328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:43:57.555837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:43:57.567758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1198/2409] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:43:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:44:04.954331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=50 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:44:07.563164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=30 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:44:22.467888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:22.468700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:22.544417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865040be-8a58-42e8-b4d7-0ea42cbf8def service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:22.549976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:22.550952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:22.550952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:22.551141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:22.553270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:22.618729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:22.619707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:22.631397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1212/2410] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:44:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:44:22.693115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:22.693115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:22.746120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9b985-6adf-4afe-b105-04995c47b539 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:22.750543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:22.752972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:22.752972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:22.752972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:22.753455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:22.768126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:22.769639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:22.781040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1199/2411] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:44:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:44:22.947508 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:22.948919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:23.007493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a8c51-1818-491a-bd03-1c8bc981ee13 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:23.011102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:23.011713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:23.012002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:23.012306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:23.014659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:23.028880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:23.030994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3087031-7f1c-4a23-bd21-65914a9a8cd8 service glance] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:23.043179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1213/2412] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:44:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:44:32.571627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:32.573292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:32.584662 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352a579-4948-44c8-913b-a635806dffdd service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:32.588248 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:32.588864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:32.589149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:32.589458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:32.591186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:32.602173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:32.606307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:32.613302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1200/2413] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:44:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:44:33.037587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=82 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:44:42.398310 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873dc04-7e37-4baa-b775-9a17ebd71979 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:42.398874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873dc04-7e37-4baa-b775-9a17ebd71979 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:42.398874 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873dc04-7e37-4baa-b775-9a17ebd71979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:42.419313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5873dc04-7e37-4baa-b775-9a17ebd71979 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:44:42.421122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5873dc04-7e37-4baa-b775-9a17ebd71979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:42.478969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1214/2414] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:44:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:44:42.518802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:42.520252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:42.590550 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285bc80-830f-4594-825c-a380c2cfed1a service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:42.596213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:42.596829 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:42.597115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:42.597427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:42.605526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:42.607433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:42.615505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1201/2415] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:44:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:44:52.471934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=25 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:44:52.612205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=64 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:44:52.696469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:52.696756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:52.697030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:52.719131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] MFA Rules not processed for user `3ec8d131a9764028a096314a4ecd58f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:44:52.721676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:52.733702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1215/2416] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:44:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:44:59.173953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:59.186299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e36b2-1794-4f35-9db6-fa1b8fc3af1e service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:59.192581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:59.193192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:59.193497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:59.193770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:59.195575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:59.277843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:59.280119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:59.292582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1202/2417] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:44:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:44:59.430506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:44:59.434569 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:59.452424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf417b-4781-4936-a843-db60fbfe6a4f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:44:59.459581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:44:59.460212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:44:59.460488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:44:59.460759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:44:59.462612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:44:59.476564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:44:59.478311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:44:59.490245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1216/2418] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:44:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:00.043209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:00.072948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433601-d4aa-43e4-bd7a-5a73b9c16eb9 service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:00.082983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:00.083851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:00.089418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:00.089657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:00.092531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:00.110076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:00.113506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27a538-7b45-4143-b7f7-888466bdbcd1 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:00.127674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1203/2419] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:45:03.028104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:03.038129 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead57d1-e90d-444b-80f6-f3ace41ea37d service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:03.041686 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:03.042283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.042562 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.042833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.044552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:03.053959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:03.055381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d6ec3-9f8f-4d98-ad31-f6ed5ebfc346 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:03.063872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1217/2420] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:03.432726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513bbd1-8944-4cdf-99e8-9ca4f4d8ebfc None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.433301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513bbd1-8944-4cdf-99e8-9ca4f4d8ebfc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.433301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513bbd1-8944-4cdf-99e8-9ca4f4d8ebfc None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.434705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1204/2421] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:45:03.442395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6c965-d318-4938-a2ce-0c7da1f3d7f6 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.442796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6c965-d318-4938-a2ce-0c7da1f3d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.443169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6c965-d318-4938-a2ce-0c7da1f3d7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.464240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c9d6c965-d318-4938-a2ce-0c7da1f3d7f6 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:03.465510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c9d6c965-d318-4938-a2ce-0c7da1f3d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:03.476160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1218/2422] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:45:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:45:03.491137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513bbd1-8944-4cdf-99e8-9ca4f4d8ebfc None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:03.502989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513bbd1-8944-4cdf-99e8-9ca4f4d8ebfc None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:03.506688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17695ca1-b9f8-44e7-b938-5679d7682186 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:03.507391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17695ca1-b9f8-44e7-b938-5679d7682186 service glance] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.507677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17695ca1-b9f8-44e7-b938-5679d7682186 service glance] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.508090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17695ca1-b9f8-44e7-b938-5679d7682186 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.519927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17695ca1-b9f8-44e7-b938-5679d7682186 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:03.521419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17695ca1-b9f8-44e7-b938-5679d7682186 service glance] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:03.533819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1205/2423] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:03.751521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06ffc4-233c-463e-9bca-c64cd18e595a None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.754506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06ffc4-233c-463e-9bca-c64cd18e595a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.754506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06ffc4-233c-463e-9bca-c64cd18e595a None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.754506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1219/2424] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 03:45:03.758080 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b9ef0-c771-4beb-b86e-f44d7e121cb4 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.758371 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b9ef0-c771-4beb-b86e-f44d7e121cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.758646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b9ef0-c771-4beb-b86e-f44d7e121cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.778411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-519b9ef0-c771-4beb-b86e-f44d7e121cb4 None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:03.779600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-519b9ef0-c771-4beb-b86e-f44d7e121cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:03.786818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1206/2425] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:45:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:45:03.800149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06ffc4-233c-463e-9bca-c64cd18e595a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:03.811253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06ffc4-233c-463e-9bca-c64cd18e595a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:03.815699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a622-5408-4f41-86fc-cdf2267a6273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:03.816307 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a622-5408-4f41-86fc-cdf2267a6273 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.816584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a622-5408-4f41-86fc-cdf2267a6273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.816853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a622-5408-4f41-86fc-cdf2267a6273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.824320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05a622-5408-4f41-86fc-cdf2267a6273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:03.825801 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05a622-5408-4f41-86fc-cdf2267a6273 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:03.835031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1220/2426] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:03.920006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f390c-ef72-4bde-a559-885662032c82 None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.920295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f390c-ef72-4bde-a559-885662032c82 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.920573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f390c-ef72-4bde-a559-885662032c82 None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.922010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1207/2427] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:45:03.927333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.927616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.927911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.947634 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:03.948761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:03.958989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1221/2428] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:45:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:45:03.969394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f390c-ef72-4bde-a559-885662032c82 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:03.980255 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f390c-ef72-4bde-a559-885662032c82 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:03.984123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68c58e-012e-447b-8b10-ee131387a891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:03.984724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68c58e-012e-447b-8b10-ee131387a891 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:03.985003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68c58e-012e-447b-8b10-ee131387a891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:03.985319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68c58e-012e-447b-8b10-ee131387a891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:03.995318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68c58e-012e-447b-8b10-ee131387a891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:04.000567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68c58e-012e-447b-8b10-ee131387a891 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:04.008061 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1208/2429] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:45:04.074901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:04.076320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:04.143949 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b51a75-6e08-4c88-ac08-d008bbfb5c59 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:04.148999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:04.149670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:04.149950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:04.150220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:04.151990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:04.163488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:04.165083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:04.175792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1222/2430] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:07.721022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881e3d-47f2-4ff6-9c79-34a29ef4fac6 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:07.721494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881e3d-47f2-4ff6-9c79-34a29ef4fac6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:07.721599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881e3d-47f2-4ff6-9c79-34a29ef4fac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:07.741477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-89881e3d-47f2-4ff6-9c79-34a29ef4fac6 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:07.742654 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-89881e3d-47f2-4ff6-9c79-34a29ef4fac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:07.750126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1209/2431] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:45:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:45:07.761856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:07.772939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4fb8a-2a73-4645-a7d5-b108958d76cd service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:07.778133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:07.778741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:07.779022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:07.779324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:07.788118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:07.789730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:07.798314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1223/2432] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:45:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:45:17.737716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=52 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:45:17.794412 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=65 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:45:38.785997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:38.786535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:38.786535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:38.807364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:38.808588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:38.815946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1210/2433] 10.222.0.66 () {58 vars in 967 bytes} [Thu Aug 4 03:45:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:45:38.828380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:38.839377 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482954e-7c2d-420e-abdf-6fb102001faa service swift] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:38.844472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149f333-827d-4391-ba02-3f1330517c6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:38.845520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149f333-827d-4391-ba02-3f1330517c6a service swift] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:38.850768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149f333-827d-4391-ba02-3f1330517c6a service swift] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:38.850768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149f333-827d-4391-ba02-3f1330517c6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:38.855210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149f333-827d-4391-ba02-3f1330517c6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:38.856853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149f333-827d-4391-ba02-3f1330517c6a service swift] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:38.865239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1224/2434] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:45:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:45:48.801547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:45:48.862357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=6 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:45:58.263674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:58.273272 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef865920-5f97-4b52-8567-013655ec9146 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:58.277083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:58.277731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.278007 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.278276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.280002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:58.290467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:58.291908 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6b7bf-3508-47a7-b9d8-cd5407f75714 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:58.299792 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1211/2435] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:58.706474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cba85-5cf4-45db-8357-577f5b3a7baf None None] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.707038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cba85-5cf4-45db-8357-577f5b3a7baf None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.707038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cba85-5cf4-45db-8357-577f5b3a7baf None None] PATH_INFO: `/` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.708482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1225/2436] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:45:58.715347 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2764-811e-4f59-a326-f99e325040f9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.715632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2764-811e-4f59-a326-f99e325040f9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.715902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2764-811e-4f59-a326-f99e325040f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.735844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3dab2764-811e-4f59-a326-f99e325040f9 None None] MFA Rules not processed for user `8c54c5182ce245e49cdbf25f1254238e`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:58.737064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3dab2764-811e-4f59-a326-f99e325040f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:58.743678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1212/2437] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:45:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:45:58.754962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cba85-5cf4-45db-8357-577f5b3a7baf None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:58.766888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cba85-5cf4-45db-8357-577f5b3a7baf None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:58.770499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fb96c-db68-41d5-a73c-ea185a5f5fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:58.771111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fb96c-db68-41d5-a73c-ea185a5f5fee service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.771512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fb96c-db68-41d5-a73c-ea185a5f5fee service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.771789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fb96c-db68-41d5-a73c-ea185a5f5fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.780265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fb96c-db68-41d5-a73c-ea185a5f5fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:58.781813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fb96c-db68-41d5-a73c-ea185a5f5fee service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:58.789641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1226/2438] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:45:58.871352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ee6f-ee2c-4f17-9650-4654f476583a None None] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.871643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ee6f-ee2c-4f17-9650-4654f476583a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.871922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ee6f-ee2c-4f17-9650-4654f476583a None None] PATH_INFO: `/` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.873390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1213/2439] 10.222.0.66 () {54 vars in 921 bytes} [Thu Aug 4 03:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 03:45:58.878654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.878936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.879211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.899770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] MFA Rules not processed for user `09fbb54e522c47d89de166682742336f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:45:58.900970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:45:58.910175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1227/2440] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Aug 4 03:45:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:45:58.920641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ee6f-ee2c-4f17-9650-4654f476583a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:45:58.931855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ee6f-ee2c-4f17-9650-4654f476583a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:45:58.935479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:45:58.936079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:45:58.936360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:45:58.936637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:45:58.947571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:45:58.949058 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:45:58.959921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1214/2441] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:45:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:46:08.894175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=9 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:08.955256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=15 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:14.318376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:14.336416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8763e4-f288-462e-8288-f8ff57ad2a11 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:14.343075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:14.343448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:14.343730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:14.344005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:14.347224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:14.360702 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:14.362353 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:14.374464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1228/2442] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:46:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:46:24.368884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=6 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:26.893330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:26.902513 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359dad3-40c5-4797-a495-df727bf0a0d9 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:26.906195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:26.906797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:26.907084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:26.907354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:26.909112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:26.971410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:26.972879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:26.983378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1215/2443] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:46:31.000628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:31.013149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f3554-6ee5-4688-b54c-89da400294fc service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:31.018072 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:31.018863 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:31.019146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:31.019416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:31.021125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:31.034030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:31.039049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:31.046036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1229/2444] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:46:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:46:31.149032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:31.150567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:31.168912 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed03448-2346-41d3-83d7-0e5f17f93a27 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:31.175632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:31.176748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:31.176748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:31.176941 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] PATH_INFO: `/v3/users/3ec8d131a9764028a096314a4ecd58f9` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:31.178108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec8d131a9764028a096314a4ecd58f9)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:31.180048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:31.373562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec8d131a9764028a096314a4ecd58f9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.374379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec8d131a9764028a096314a4ecd58f9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.374981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec8d131a9764028a096314a4ecd58f9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.389821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec8d131a9764028a096314a4ecd58f9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.451306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-98966619-740e-4a1f-8a2a-000806abdcdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec8d131a9764028a096314a4ecd58f9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.467154 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1216/2445] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:46:31 2022] DELETE /identity/v3/users/3ec8d131a9764028a096314a4ecd58f9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:46:31.721102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:31.734583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2f418-4dd0-4f81-bf85-21d0e9d9353e service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:31.795918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:31.796682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:31.796961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:31.797286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] PATH_INFO: `/v3/projects/7d9ff6952b014d0a9abb28dcf0b818b0` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:31.798523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9ff6952b014d0a9abb28dcf0b818b0)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:31.800536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:31.903627 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9ff6952b014d0a9abb28dcf0b818b0', 'request_id': 'req-be3b669d-448a-406a-bcee-310e6cb434b8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.909827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9ff6952b014d0a9abb28dcf0b818b0', 'request_id': 'req-be3b669d-448a-406a-bcee-310e6cb434b8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.988114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9ff6952b014d0a9abb28dcf0b818b0', 'request_id': 'req-be3b669d-448a-406a-bcee-310e6cb434b8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:31.999707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] The token cache is being invalidate because project 7d9ff6952b014d0a9abb28dcf0b818b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:46:32.003430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:32.003430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-be3b669d-448a-406a-bcee-310e6cb434b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:46:32.003430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1230/2446] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:46:31 2022] DELETE /identity/v3/projects/7d9ff6952b014d0a9abb28dcf0b818b0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:46:32.007240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.007533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.007811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.029458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:32.030665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:32.095753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1217/2447] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:32.103522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.103809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.104346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.126881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:32.126881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:32.134714 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1231/2448] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:32.139766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.149669 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc70a67-a1da-41a7-945d-feada3e76754 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.154567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.155198 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.155480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.155755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.156452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:46:32.156760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.158397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.159889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:32.171057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1218/2449] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:46:32.176001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.186740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12126eff-adae-45b0-ad5c-23e2693efa9e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.191434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.192038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.192313 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.192583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.193271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.195189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.252802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1232/2450] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:46:32 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:46:32.256910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.267500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670d434-39ed-42f8-a7d8-4ce79d8d2a95 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.323943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.324564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.324861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.325134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.325837 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.327736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.440795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1219/2451] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:46:32 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:46:32.444849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.455571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46008f6c-0570-4871-a668-a00d09569d86 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.460229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.460832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.461176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.461417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.462112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.463843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.473884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1233/2452] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:46:32.477950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.488684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9f6e6-b6ee-438a-b235-07cdfbc91335 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.493443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e90921-dc80-4bd2-a5f8-e29f4a3e52c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.494114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e90921-dc80-4bd2-a5f8-e29f4a3e52c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.494466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e90921-dc80-4bd2-a5f8-e29f4a3e52c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.495437 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e90921-dc80-4bd2-a5f8-e29f4a3e52c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.496294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e90921-dc80-4bd2-a5f8-e29f4a3e52c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.498011 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e90921-dc80-4bd2-a5f8-e29f4a3e52c2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.508409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1220/2453] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:46:32.512570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.522347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ab229-6b51-4196-b784-85a2af90dc7c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.526973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.527610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.527887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.528156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] PATH_INFO: `/v3/projects/a98f18ee8c7d4cfe9f4b879505ca6605/users/ac99f60384fe4a119d019cedfe1d3557/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.542847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98f18ee8c7d4cfe9f4b879505ca6605, user_id=ac99f60384fe4a119d019cedfe1d3557, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.545544 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.628456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1234/2454] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:46:32 2022] PUT /identity/v3/projects/a98f18ee8c7d4cfe9f4b879505ca6605/users/ac99f60384fe4a119d019cedfe1d3557/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:46:32.633755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf2257-6c41-4587-857b-0f94434aff89 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.634073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf2257-6c41-4587-857b-0f94434aff89 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.634359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf2257-6c41-4587-857b-0f94434aff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.653716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1dcf2257-6c41-4587-857b-0f94434aff89 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:32.654938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1dcf2257-6c41-4587-857b-0f94434aff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:32.712823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1221/2455] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:32.722360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:32.724056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:32.739251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:32.781860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5a57b-f12a-4b27-8166-69fee4ca9601 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:32.785502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:32.786111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:32.786391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:32.786667 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:32.796280 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:32.797830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:32.807141 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1235/2456] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:46:39.283206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77596e1c-06aa-40a2-b0e4-55bda17c66ba None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:39.284680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77596e1c-06aa-40a2-b0e4-55bda17c66ba None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:39.285402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77596e1c-06aa-40a2-b0e4-55bda17c66ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:39.321184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-77596e1c-06aa-40a2-b0e4-55bda17c66ba None None] MFA Rules not processed for user `ac99f60384fe4a119d019cedfe1d3557`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:39.322680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-77596e1c-06aa-40a2-b0e4-55bda17c66ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:39.426805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1222/2457] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:39.442778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:39.446173 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:39.594748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaac225-20df-49d8-8ce7-3c8ee021a2a7 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:39.604325 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2629b-80c8-4b5d-ae2a-54f0cbcce35a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:39.605188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2629b-80c8-4b5d-ae2a-54f0cbcce35a service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:39.605653 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2629b-80c8-4b5d-ae2a-54f0cbcce35a service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:39.606075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2629b-80c8-4b5d-ae2a-54f0cbcce35a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:39.621013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2629b-80c8-4b5d-ae2a-54f0cbcce35a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:39.622977 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2629b-80c8-4b5d-ae2a-54f0cbcce35a service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:39.634689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1236/2458] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:46:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:46:39.772830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a29a2-7d66-4136-9b1e-702add85a79d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:39.773764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a29a2-7d66-4136-9b1e-702add85a79d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:39.774372 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a29a2-7d66-4136-9b1e-702add85a79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:39.783318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:39.797576 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c99a29a2-7d66-4136-9b1e-702add85a79d None None] MFA Rules not processed for user `6dba436d0a67433f8c8ae7cd40d3f795`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:39.798878 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c99a29a2-7d66-4136-9b1e-702add85a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:39.864857 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1223/2459] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:40.450239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:40.450239 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:40.451511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:40.486564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] MFA Rules not processed for user `ac48e585c133487dadbe0a1e095bfde0`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:40.488178 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:40.573488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1237/2460] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:42.473529 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc847-6625-4dae-9ae5-7ab9fe065f74 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:42.474435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc847-6625-4dae-9ae5-7ab9fe065f74 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:42.474716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bc847-6625-4dae-9ae5-7ab9fe065f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:42.496923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7d4bc847-6625-4dae-9ae5-7ab9fe065f74 None None] MFA Rules not processed for user `ac99f60384fe4a119d019cedfe1d3557`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:46:42.499187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7d4bc847-6625-4dae-9ae5-7ab9fe065f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:42.510051 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1224/2461] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:46:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:46:47.434865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:46:47.436302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:47.453137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216feb7-6e8a-46d1-b599-cdc03918a8d4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:46:47.459252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:46:47.459872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:46:47.460148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:46:47.460418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:46:47.462188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:46:47.523238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:47.538920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:46:47.541435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:46:47.551700 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1238/2462] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:46:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:46:52.494764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=27 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:46:57.553517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:47:01.926591 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef4f9-45f7-4509-a1de-d894a9d3b4dc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:01.927076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef4f9-45f7-4509-a1de-d894a9d3b4dc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:01.927169 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef4f9-45f7-4509-a1de-d894a9d3b4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:01.950194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-fb4ef4f9-45f7-4509-a1de-d894a9d3b4dc None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:47:01.950194 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fb4ef4f9-45f7-4509-a1de-d894a9d3b4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:02.029087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1225/2463] 10.222.0.66 () {58 vars in 967 bytes} [Thu Aug 4 03:47:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:47:02.239250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:02.239250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:02.326226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d8aad-78ae-4908-a722-72b79d640fb7 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:02.337057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:02.338205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:02.338815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:02.339314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:02.350675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:02.352739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:02.363549 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1239/2464] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:47:06.331569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:06.332109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:06.332109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:06.351267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] MFA Rules not processed for user `55beb38ea2994ea7987fe546235b58a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:47:06.352484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:06.362144 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1226/2465] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:47:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:47:12.361456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=32 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:47:14.289986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:14.291454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:14.413494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:14.415239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:14.435268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd87aa2b-152b-4f27-a6b4-0994c2b760af service swift] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:14.441453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:14.442082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:14.442365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:14.442638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:14.444616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:14.457368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:14.459156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:14.469439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1240/2466] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:47:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:47:14.494033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015965-3b26-469e-a95c-3117cb0568cf None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:14.499055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:14.499492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:14.499779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:14.500079 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] PATH_INFO: `/v3/users/d245c731eb58483bb635a7e70542b892` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:14.501288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d245c731eb58483bb635a7e70542b892)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:14.503234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:14.679844 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd245c731eb58483bb635a7e70542b892'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:14.681419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd245c731eb58483bb635a7e70542b892'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:14.682329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd245c731eb58483bb635a7e70542b892'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:14.699386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd245c731eb58483bb635a7e70542b892'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:14.770525 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd245c731eb58483bb635a7e70542b892'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:14.785186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1227/2467] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:47:14 2022] DELETE /identity/v3/users/d245c731eb58483bb635a7e70542b892 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:47:15.041402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:15.067565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d256f-6391-4b64-ab0a-8420bb45cef1 service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:15.136089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:15.147579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:15.147579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:15.147579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] PATH_INFO: `/v3/projects/2049d7705d5c4afe94c22a6f1284327b` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:15.147579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2049d7705d5c4afe94c22a6f1284327b)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:15.147579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:15.167868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:47:15.246739 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2049d7705d5c4afe94c22a6f1284327b', 'request_id': 'req-ba9ec73a-fbb6-4344-969a-364e714facbc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.253745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2049d7705d5c4afe94c22a6f1284327b', 'request_id': 'req-ba9ec73a-fbb6-4344-969a-364e714facbc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.330709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2049d7705d5c4afe94c22a6f1284327b', 'request_id': 'req-ba9ec73a-fbb6-4344-969a-364e714facbc'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.342931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] The token cache is being invalidate because project 2049d7705d5c4afe94c22a6f1284327b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:47:15.343235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.343899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.345469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1241/2468] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:47:15 2022] DELETE /identity/v3/projects/2049d7705d5c4afe94c22a6f1284327b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:47:15.349680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:15.351172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:15.365617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a253a8-b465-4140-b2a1-b771c6db1e71 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:15.379404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:47:15.441345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:15.442000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:15.442206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:15.442495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] PATH_INFO: `/v3/users/cf10316294aa44919aae74d21819af3c` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:15.443751 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf10316294aa44919aae74d21819af3c)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:15.445750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:15.670173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf10316294aa44919aae74d21819af3c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.671395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf10316294aa44919aae74d21819af3c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.672015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf10316294aa44919aae74d21819af3c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.684730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf10316294aa44919aae74d21819af3c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.753467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf10316294aa44919aae74d21819af3c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:15.767734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1228/2469] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:47:15 2022] DELETE /identity/v3/users/cf10316294aa44919aae74d21819af3c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:47:16.042399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:16.043996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:16.059032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ec73a-fbb6-4344-969a-364e714facbc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:16.123273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:16.124162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:16.124448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:16.124728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] PATH_INFO: `/v3/projects/a0164049927c4e658615cd0cba7a4774` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:16.125931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0164049927c4e658615cd0cba7a4774)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:16.128010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:16.248281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0164049927c4e658615cd0cba7a4774', 'request_id': 'req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:16.255071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0164049927c4e658615cd0cba7a4774', 'request_id': 'req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:16.313249 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0164049927c4e658615cd0cba7a4774', 'request_id': 'req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:16.324616 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] The token cache is being invalidate because project a0164049927c4e658615cd0cba7a4774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:47:16.325011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:16.325692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:47:16.330076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1242/2470] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:47:16 2022] DELETE /identity/v3/projects/a0164049927c4e658615cd0cba7a4774 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:47:25.762842 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=46,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:47:26.321378 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:47:55.215911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:55.217380 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:55.293648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db00c3c-f0c7-4c03-8c39-6c257fd2c9a3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:47:55.340631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:47:55.341260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:47:55.341541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:47:55.341807 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:47:55.343517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:47:55.400358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:47:55.401864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:47:55.410747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1229/2471] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:47:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:47:59.946577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:47:59.948115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:00.017919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8da2e-2423-4d9f-88b5-f97bd4ccfb1e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:48:00.022823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:48:00.023441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:00.023722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:00.024532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:00.026457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:00.038085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:48:00.040335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:48:00.050420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1243/2472] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:47:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:48:01.733141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:48:01.735753 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:01.745797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c027fa-a5ba-4184-9a58-22cbe1706ee0 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:48:01.749441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:48:01.750042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:01.750324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:01.750598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:01.752282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:01.800729 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:48:01.815621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:48:01.817143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:48:01.827443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1230/2473] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:48:09.062030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:48:09.063476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:09.078773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a1dc0-861a-40d7-ae5d-7ba6e1b58dde service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:48:09.083150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:48:09.083774 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:09.084054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:09.084326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] PATH_INFO: `/v3/users/dcc12077b11448a3b1ec2e7dbf2707ef` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:09.085554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcc12077b11448a3b1ec2e7dbf2707ef)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:48:09.088474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:48:09.234457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcc12077b11448a3b1ec2e7dbf2707ef'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.241154 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcc12077b11448a3b1ec2e7dbf2707ef'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.241831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcc12077b11448a3b1ec2e7dbf2707ef'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.253587 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcc12077b11448a3b1ec2e7dbf2707ef'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.300580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcc12077b11448a3b1ec2e7dbf2707ef'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.307579 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1244/2474] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:48:09 2022] DELETE /identity/v3/users/dcc12077b11448a3b1ec2e7dbf2707ef => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:48:09.553703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:48:09.564505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b835263-caa3-440b-a872-4da5aa6d3647 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:48:09.621689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:48:09.622333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:09.622615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:09.622890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] PATH_INFO: `/v3/projects/cf95e3e43d0e482ba4d27b769fb48628` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:09.624070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf95e3e43d0e482ba4d27b769fb48628)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:48:09.626112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:48:09.730190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf95e3e43d0e482ba4d27b769fb48628', 'request_id': 'req-71735ec0-c1fa-4497-a694-c48d581ea738'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.736526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf95e3e43d0e482ba4d27b769fb48628', 'request_id': 'req-71735ec0-c1fa-4497-a694-c48d581ea738'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.815050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf95e3e43d0e482ba4d27b769fb48628', 'request_id': 'req-71735ec0-c1fa-4497-a694-c48d581ea738'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.826361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] The token cache is being invalidate because project cf95e3e43d0e482ba4d27b769fb48628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:48:09.826650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.827276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:09.828790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1231/2475] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:48:09 2022] DELETE /identity/v3/projects/cf95e3e43d0e482ba4d27b769fb48628 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:48:09.833043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:48:09.834502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:09.847480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b64aae-a2d9-4096-af33-d36585b20780 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:48:09.902756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:48:09.903386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:09.903664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:09.903934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] PATH_INFO: `/v3/users/a60bf4f434b74d47b0c73c60177d101c` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:09.905099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a60bf4f434b74d47b0c73c60177d101c)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:48:09.907055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:48:09.909314 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:48:10.118654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a60bf4f434b74d47b0c73c60177d101c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.125350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a60bf4f434b74d47b0c73c60177d101c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.126016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a60bf4f434b74d47b0c73c60177d101c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.138904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a60bf4f434b74d47b0c73c60177d101c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.202768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-145339aa-171a-4a2c-9b87-5a75c93aa7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a60bf4f434b74d47b0c73c60177d101c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.209040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1245/2476] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:48:09 2022] DELETE /identity/v3/users/a60bf4f434b74d47b0c73c60177d101c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:48:10.427865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:48:10.429438 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:10.443891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71735ec0-c1fa-4497-a694-c48d581ea738 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:48:10.504823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:48:10.505557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:10.505841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:10.506119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] PATH_INFO: `/v3/projects/10d1be733d35463181aea92f1fcfa3f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:10.507299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d1be733d35463181aea92f1fcfa3f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:48:10.509954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:48:10.526325 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:48:10.604350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d1be733d35463181aea92f1fcfa3f8', 'request_id': 'req-d34fa0ad-a261-4460-9d30-4669d44fbf15'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.610886 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d1be733d35463181aea92f1fcfa3f8', 'request_id': 'req-d34fa0ad-a261-4460-9d30-4669d44fbf15'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.666189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d1be733d35463181aea92f1fcfa3f8', 'request_id': 'req-d34fa0ad-a261-4460-9d30-4669d44fbf15'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.679445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] The token cache is being invalidate because project 10d1be733d35463181aea92f1fcfa3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:48:10.679445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.679445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:48:10.679445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1232/2477] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:48:10 2022] DELETE /identity/v3/projects/10d1be733d35463181aea92f1fcfa3f8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:48:20.214052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:48:20.673727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:48:51.011631 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053d22f-2a01-41bc-aedb-c2c966306c73 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:48:51.012180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053d22f-2a01-41bc-aedb-c2c966306c73 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:48:51.012180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053d22f-2a01-41bc-aedb-c2c966306c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:48:51.031553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8053d22f-2a01-41bc-aedb-c2c966306c73 None None] MFA Rules not processed for user `ac99f60384fe4a119d019cedfe1d3557`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:48:51.032768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8053d22f-2a01-41bc-aedb-c2c966306c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:48:51.097240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1246/2478] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:48:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:01.090480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=25 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:07.975838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:07.977342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:08.064436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa0ad-a261-4460-9d30-4669d44fbf15 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:08.068507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:08.069142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:08.069667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:08.069948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:08.071926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:08.172962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:08.174677 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:08.188519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1233/2479] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:17.150492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382274b-410e-4e4f-a4bc-579976d59e7c None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:17.151501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382274b-410e-4e4f-a4bc-579976d59e7c None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:17.151771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382274b-410e-4e4f-a4bc-579976d59e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:17.186752 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9382274b-410e-4e4f-a4bc-579976d59e7c None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:17.188618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9382274b-410e-4e4f-a4bc-579976d59e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:17.253240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1247/2480] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:49:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:17.372037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:17.377101 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:17.447970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831857b8-0944-48ca-9617-01226fb1cafb service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:17.452964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:17.453632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:17.453920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:17.454225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:17.462848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:17.464588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:17.473397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1234/2481] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:49:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:49:21.069457 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:21.069991 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:21.070093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:21.092443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] MFA Rules not processed for user `ac48e585c133487dadbe0a1e095bfde0`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:21.094040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:21.105633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1248/2482] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:21.686103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:21.688297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:21.722245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:21.772311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ce1c7-f291-47e8-8233-41020ecfeed3 service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:21.776973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:21.777821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:21.778209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:21.778613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:21.790321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:21.879949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:21.881413 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:21.897862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1235/2483] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:21.998475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:22.000972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:22.020672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:22.022141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:22.023814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf75ef-f7d5-4fa3-a5f8-8fde0e4c8743 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:22.027974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:22.028918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:22.031467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:22.031467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:22.034994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817c7-b914-4cf5-9819-3781623422be service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:22.035308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:22.039143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:22.039333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:22.039573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:22.039852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:22.041734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:22.138987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:22.141880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:22.144063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:22.148216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:22.175079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1249/2484] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:22.185353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1236/2485] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:49:27.158853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:27.161044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:27.176799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622b635-85ad-46ed-a204-c0bc3d6b23ae service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:27.181684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:27.184747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:27.185035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:27.185342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] PATH_INFO: `/v3/users/6dba436d0a67433f8c8ae7cd40d3f795` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:27.201376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dba436d0a67433f8c8ae7cd40d3f795)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:27.203366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:27.585673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:27.602826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dba436d0a67433f8c8ae7cd40d3f795'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:27.609881 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dba436d0a67433f8c8ae7cd40d3f795'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:27.610591 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dba436d0a67433f8c8ae7cd40d3f795'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:27.630613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dba436d0a67433f8c8ae7cd40d3f795'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:27.704471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-5e596655-ec34-49de-8b1a-580499edc33c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dba436d0a67433f8c8ae7cd40d3f795'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:27.711919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1250/2486] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:49:27 2022] DELETE /identity/v3/users/6dba436d0a67433f8c8ae7cd40d3f795 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:28.139659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:28.157655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc5482-b68d-41fe-8fb1-4d656c1f8b35 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:28.246537 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:28.269419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:28.270110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:28.270398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:28.270750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] PATH_INFO: `/v3/projects/f46b8dc64007429b805971d4c6f5b471` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:28.285292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46b8dc64007429b805971d4c6f5b471)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:28.287692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:28.411631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46b8dc64007429b805971d4c6f5b471', 'request_id': 'req-a2022280-1dcd-4f97-83d8-e2520436445e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:28.430076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46b8dc64007429b805971d4c6f5b471', 'request_id': 'req-a2022280-1dcd-4f97-83d8-e2520436445e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:28.522469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46b8dc64007429b805971d4c6f5b471', 'request_id': 'req-a2022280-1dcd-4f97-83d8-e2520436445e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:28.549137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] The token cache is being invalidate because project f46b8dc64007429b805971d4c6f5b471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:49:28.549544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:28.553639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a2022280-1dcd-4f97-83d8-e2520436445e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:28.559203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1237/2487] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:49:28 2022] DELETE /identity/v3/projects/f46b8dc64007429b805971d4c6f5b471 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:28.562969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:28.563887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:28.564421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:28.653680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:28.655445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:28.749233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1251/2488] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:49:28.758714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:28.759004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:28.759283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:28.790220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:28.791957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:28.803587 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1238/2489] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:28.804931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:28.824119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b94cd-1e28-4a8a-9152-d2d218ed0183 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:28.829027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:28.829689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:28.829973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:28.830537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:28.831348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:49:28.831762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:28.833576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:28.847188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1252/2490] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:49:28.851693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:28.862994 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c37f4a-9e5b-42e2-8d63-3e43bf36ffa9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:28.870227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:28.871187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:28.871810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:28.872463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:28.873407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:28.876568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.016810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1239/2491] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:49:28 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:29.021255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:29.040362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb521e4-2dd0-4f68-b767-e8186c0693cd admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:29.131081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:29.132885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.133265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.133654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.135813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:29.136424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.152499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:29.246950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1253/2492] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:49:29 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:29.248196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:29.263162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce09ce-35fa-4294-8725-da6b67aa9dd4 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:29.275500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:29.276304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.276783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.277013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.278376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:29.282377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.313897 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1240/2493] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:29.319264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:29.331677 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a2cb3-79be-4c6d-b140-0f8a8bb2dbe3 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:29.338894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e21ba6-cfaf-4762-9fcd-5f4572d19c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:29.339877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e21ba6-cfaf-4762-9fcd-5f4572d19c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.340162 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e21ba6-cfaf-4762-9fcd-5f4572d19c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.340468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e21ba6-cfaf-4762-9fcd-5f4572d19c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.341275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e21ba6-cfaf-4762-9fcd-5f4572d19c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:29.342940 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e21ba6-cfaf-4762-9fcd-5f4572d19c7d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.355592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1254/2494] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:29.359731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:29.371083 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39faf-dd88-4136-ace6-f5b937d589b9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:29.375777 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:29.376473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.376716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.377020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] PATH_INFO: `/v3/projects/5e946124639844679b627087746232b9/users/409c1132b2dd46349950d456a2d2deb6/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.395718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e946124639844679b627087746232b9, user_id=409c1132b2dd46349950d456a2d2deb6, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:29.398698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.466935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1241/2495] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:29 2022] PUT /identity/v3/projects/5e946124639844679b627087746232b9/users/409c1132b2dd46349950d456a2d2deb6/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:29.485910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.486194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.487340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.527070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] MFA Rules not processed for user `409c1132b2dd46349950d456a2d2deb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:29.528210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:29.610014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1255/2496] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:29.616064 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:29.674316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:29.689187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ff721-5f4f-4519-bb84-74fa2188de06 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:29.695896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:29.696813 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.697165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.697657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.698426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:29.700418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.782349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1242/2497] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:49:29 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:29.784748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:29.809867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f15ee-31fc-4f50-a9f1-9e5dd67da781 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:29.919981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:29.921291 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:29.921780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:29.922252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:29.923127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:29.925916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:29.999716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1256/2498] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:49:29 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:30.004330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:30.017616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c703c-8238-4fc9-8661-969a5c5c4b01 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:30.023185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:30.023978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.024327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.024675 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.025540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:30.027342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:30.043697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1243/2499] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:49:30.045253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:30.056925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa1547-ecb3-4868-9634-23bfd9756521 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:30.061781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:30.062555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.062885 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.063257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] PATH_INFO: `/v3/projects/394221b77b9145fda0b10ab0a19671b2/users/38514e975fcc4714b60e9ff85dd87277/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.083783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=394221b77b9145fda0b10ab0a19671b2, user_id=38514e975fcc4714b60e9ff85dd87277, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:30.087820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:30.167217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1257/2500] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:30 2022] PUT /identity/v3/projects/394221b77b9145fda0b10ab0a19671b2/users/38514e975fcc4714b60e9ff85dd87277/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:49:30.171295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:30.185793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd21c9d-c9c3-4c7f-91d8-a29700f6aa95 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:30.257745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8abb2-c7b8-44d3-aa38-2ad393846c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:30.258457 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8abb2-c7b8-44d3-aa38-2ad393846c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.258664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8abb2-c7b8-44d3-aa38-2ad393846c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.258938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8abb2-c7b8-44d3-aa38-2ad393846c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.259686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8abb2-c7b8-44d3-aa38-2ad393846c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:30.261534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8abb2-c7b8-44d3-aa38-2ad393846c92 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:30.277067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1244/2501] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:30.279461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:30.290966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc8067-afd1-464a-96f7-efe37bdbbd8b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:30.295857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:30.296502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.296777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.297045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] PATH_INFO: `/v3/projects/394221b77b9145fda0b10ab0a19671b2/users/38514e975fcc4714b60e9ff85dd87277/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.299629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=394221b77b9145fda0b10ab0a19671b2, user_id=38514e975fcc4714b60e9ff85dd87277, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:30.302383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:30.325493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.325771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.326511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.328716 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1258/2502] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:30 2022] PUT /identity/v3/projects/394221b77b9145fda0b10ab0a19671b2/users/38514e975fcc4714b60e9ff85dd87277/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:30.364841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] MFA Rules not processed for user `38514e975fcc4714b60e9ff85dd87277`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:30.364841 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:30.446331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1245/2503] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:30.460038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:30.461815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:30.596329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:30.596329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2c434-628c-48de-9406-73827e5941ec admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:30.596329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:30.596329 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.598009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.598009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.598009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:30.598009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:30.598009 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1259/2504] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:30.887488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:30.898300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3d45-b1c5-4e76-942d-75798c4da50c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:30.902267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ed1a-8b14-4179-a130-f24bac1b19ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:30.903278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ed1a-8b14-4179-a130-f24bac1b19ec service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:30.903608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ed1a-8b14-4179-a130-f24bac1b19ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:30.904073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ed1a-8b14-4179-a130-f24bac1b19ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:30.960818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:30.981904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ed1a-8b14-4179-a130-f24bac1b19ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:30.984035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5ed1a-8b14-4179-a130-f24bac1b19ec service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:31.008073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1246/2505] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:31.560946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:31.562552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:31.625346 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b31e13-fe6d-4cb9-a517-014d03a4439b service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:31.630081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:31.632387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:31.632387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:31.632387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:31.633754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:31.644812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:31.646417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:31.655723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1260/2506] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:41.262469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=8 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:41.651406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=40 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:42.932153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:42.932601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:42.932721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:42.982688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:42.983883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:43.048920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1247/2507] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:49:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:43.062602 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:43.064006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:43.153842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0122f-afdb-4fd2-a2c6-f456f28458f3 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:43.160167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8c90d-8c69-4017-82b1-8c41b9250335 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:43.161103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c90d-8c69-4017-82b1-8c41b9250335 service swift] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:43.161438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c90d-8c69-4017-82b1-8c41b9250335 service swift] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:43.161800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8c90d-8c69-4017-82b1-8c41b9250335 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:43.175094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c90d-8c69-4017-82b1-8c41b9250335 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:43.177093 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8c90d-8c69-4017-82b1-8c41b9250335 service swift] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:43.185969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1261/2508] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:49:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:49:44.929672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:45.000904 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c19a8c-5322-416d-b447-6c36bd53d27a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:45.008166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:45.008828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:45.009113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:45.009426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] PATH_INFO: `/v3/users/409c1132b2dd46349950d456a2d2deb6` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:45.010770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=409c1132b2dd46349950d456a2d2deb6)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:45.012801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:45.180280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '409c1132b2dd46349950d456a2d2deb6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:45.180926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '409c1132b2dd46349950d456a2d2deb6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:45.181562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '409c1132b2dd46349950d456a2d2deb6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:45.196463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '409c1132b2dd46349950d456a2d2deb6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:45.259378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '409c1132b2dd46349950d456a2d2deb6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:45.261798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:45.274443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1248/2509] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:49:44 2022] DELETE /identity/v3/users/409c1132b2dd46349950d456a2d2deb6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:45.279061 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:45.279546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:45.279663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:45.300379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:45.301682 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:45.371148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1262/2510] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:45.380949 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:45.382404 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:45.453786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b6f94a-a2a0-443f-a9a5-d5c17eb2ea4a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:45.459308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:45.460000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:45.460431 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:45.460714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:45.471521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:45.473689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:45.484298 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1249/2511] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:45.945972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:45.947814 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:45.968324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74731688-8ad0-4e35-8d44-942d3007bec4 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:45.976485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:45.977150 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:45.977473 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:45.977698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] PATH_INFO: `/v3/projects/5e946124639844679b627087746232b9` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:45.979062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e946124639844679b627087746232b9)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:45.981513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:46.088963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e946124639844679b627087746232b9', 'request_id': 'req-4e0ddf25-2cb1-4650-9971-c9048b99018e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.095214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e946124639844679b627087746232b9', 'request_id': 'req-4e0ddf25-2cb1-4650-9971-c9048b99018e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.156101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e946124639844679b627087746232b9', 'request_id': 'req-4e0ddf25-2cb1-4650-9971-c9048b99018e'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.160802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:46.182181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] The token cache is being invalidate because project 5e946124639844679b627087746232b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:49:46.182668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.183614 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.185767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1263/2512] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:49:45 2022] DELETE /identity/v3/projects/5e946124639844679b627087746232b9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:46.189765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:46.200795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284155-0c5e-4017-9fb8-b5eb73b39aa8 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:46.278890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:46.281251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:46.281251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:46.281251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] PATH_INFO: `/v3/users/38514e975fcc4714b60e9ff85dd87277` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:46.282373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=38514e975fcc4714b60e9ff85dd87277)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:46.284992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:46.493950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38514e975fcc4714b60e9ff85dd87277'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.494633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38514e975fcc4714b60e9ff85dd87277'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.495249 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38514e975fcc4714b60e9ff85dd87277'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.523935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38514e975fcc4714b60e9ff85dd87277'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.586207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90333a92-12cd-48f8-87fe-8d84069886fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38514e975fcc4714b60e9ff85dd87277'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:46.602130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1250/2513] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:49:46 2022] DELETE /identity/v3/users/38514e975fcc4714b60e9ff85dd87277 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:47.281269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:47.281269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:47.296111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ddf25-2cb1-4650-9971-c9048b99018e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:47.356758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:47.357630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:47.358014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:47.358390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] PATH_INFO: `/v3/projects/394221b77b9145fda0b10ab0a19671b2` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:47.359744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=394221b77b9145fda0b10ab0a19671b2)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:47.361855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:47.489182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394221b77b9145fda0b10ab0a19671b2', 'request_id': 'req-310e3e40-a934-42aa-b8e8-85e9516719f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:47.495617 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394221b77b9145fda0b10ab0a19671b2', 'request_id': 'req-310e3e40-a934-42aa-b8e8-85e9516719f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:47.563365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394221b77b9145fda0b10ab0a19671b2', 'request_id': 'req-310e3e40-a934-42aa-b8e8-85e9516719f5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:47.580535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] The token cache is being invalidate because project 394221b77b9145fda0b10ab0a19671b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:49:47.580901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:47.581683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-310e3e40-a934-42aa-b8e8-85e9516719f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:47.586768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1264/2514] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:49:47 2022] DELETE /identity/v3/projects/394221b77b9145fda0b10ab0a19671b2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:47.588572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:47.589050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:47.589387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:47.623793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:47.625382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:47.635929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:47.694859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1251/2515] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:47.702618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:47.702899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:47.703181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:47.728299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:47.729986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:47.744316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1265/2516] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:47.745580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:47.756323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc3338-f482-4d96-a9ee-b14e6b93633a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:47.761800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:47.762254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:47.762629 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:47.763003 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:47.764087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:49:47.764589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:47.766666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:47.790480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1252/2517] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:49:47.791755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:47.802273 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792186f-7e4e-4489-82eb-6b3a6366b88e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:47.807201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:47.807923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:47.808214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:47.808496 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:47.809174 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:47.811139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:47.912685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1266/2518] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:49:47 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:47.917166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:47.928370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec73d4f-f6a1-4dd6-bd79-25e18ca143fd admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:47.998322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:47.998871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:47.999177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:47.999450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.000121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.002107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.124417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1253/2519] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:49:47 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:48.128583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.144735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c3db7-2428-428e-9748-5fa1ea3b5fd4 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.149778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.153387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.153387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.153387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.153387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.153387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.181777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1267/2520] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:48.186151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.201250 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c4967-9159-47d3-8e1e-3c0b814a5bba admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.209027 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4294168-df34-4b7e-8b0e-b77b4b0213fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.210056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4294168-df34-4b7e-8b0e-b77b4b0213fd admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.210561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4294168-df34-4b7e-8b0e-b77b4b0213fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.211056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4294168-df34-4b7e-8b0e-b77b4b0213fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.212077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4294168-df34-4b7e-8b0e-b77b4b0213fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.214185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4294168-df34-4b7e-8b0e-b77b4b0213fd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.226877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1254/2521] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:48.231319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.241368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6acc1-facf-4153-a24b-19d1fb2d9b86 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.245907 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.246563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.246842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.247114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] PATH_INFO: `/v3/projects/2bbd6f9f5bd54142b4acb1973b15862d/users/e76f0dbda87443cf8f2612c7208b98ff/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.264287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bbd6f9f5bd54142b4acb1973b15862d, user_id=e76f0dbda87443cf8f2612c7208b98ff, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.267628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.329408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1268/2522] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:48 2022] PUT /identity/v3/projects/2bbd6f9f5bd54142b4acb1973b15862d/users/e76f0dbda87443cf8f2612c7208b98ff/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:49:48.331126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.331420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.331704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.383724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] MFA Rules not processed for user `e76f0dbda87443cf8f2612c7208b98ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:48.385610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:48.459335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1255/2523] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:48.463908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.488672 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:48.531357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ed195-a999-402b-8036-9a2fab7acfbc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.537049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.537794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.538075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.538345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.539018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.540987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.638447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1269/2524] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:49:48 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:48.642681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.655206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021920e0-d05e-45ea-a7b5-0a065a24be79 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.660339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:48.716156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.716916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.717232 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.717502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.718171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.720176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.829610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1256/2525] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:49:48 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:48.829822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.845963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0e737-9736-4789-82b8-e9d4adceb859 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.851208 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.851842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.852127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.852404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.853101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.855137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.867404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1270/2526] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:49:48.871633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.886671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bef4-29d8-460c-ac6f-83fadbcaf5f8 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:48.891486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:48.892318 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:48.892612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:48.892893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] PATH_INFO: `/v3/projects/c91726a9ffc348128a4e9586c6cddfdb/users/1b14e158c16b4c719fef005bcd1c5a6f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:48.912173 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91726a9ffc348128a4e9586c6cddfdb, user_id=1b14e158c16b4c719fef005bcd1c5a6f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:48.915251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:48.974786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1257/2527] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:48 2022] PUT /identity/v3/projects/c91726a9ffc348128a4e9586c6cddfdb/users/1b14e158c16b4c719fef005bcd1c5a6f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:48.978905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:48.991668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891829-27b2-4a50-8c88-ed515e1c12d8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:49.062664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483e31a-704c-492d-9bd7-023ee2d305a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:49.063284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483e31a-704c-492d-9bd7-023ee2d305a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:49.063585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483e31a-704c-492d-9bd7-023ee2d305a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:49.063807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483e31a-704c-492d-9bd7-023ee2d305a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:49.064507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483e31a-704c-492d-9bd7-023ee2d305a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:49.066257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483e31a-704c-492d-9bd7-023ee2d305a1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:49.105721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1271/2528] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:49.110048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:49.122109 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4572c-f3cc-4df0-9932-bb9f81d91d79 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:49.127209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:49.129604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:49.129604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:49.129604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] PATH_INFO: `/v3/projects/c91726a9ffc348128a4e9586c6cddfdb/users/1b14e158c16b4c719fef005bcd1c5a6f/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:49.131218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91726a9ffc348128a4e9586c6cddfdb, user_id=1b14e158c16b4c719fef005bcd1c5a6f, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:49.133945 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:49.145303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1258/2529] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:49 2022] PUT /identity/v3/projects/c91726a9ffc348128a4e9586c6cddfdb/users/1b14e158c16b4c719fef005bcd1c5a6f/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:49.156670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:49.156953 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:49.157247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:49.189295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] MFA Rules not processed for user `1b14e158c16b4c719fef005bcd1c5a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:49.190590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:49.274082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1272/2530] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:49.286963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:49.288682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:49.356190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15afc5-21d1-4dbe-9332-42d6d17aa806 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:49.359881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fad2a8-312a-4e30-bcb0-a7fc19c0ae5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:49.360564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad2a8-312a-4e30-bcb0-a7fc19c0ae5d service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:49.360853 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad2a8-312a-4e30-bcb0-a7fc19c0ae5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:49.361127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad2a8-312a-4e30-bcb0-a7fc19c0ae5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:49.372005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fad2a8-312a-4e30-bcb0-a7fc19c0ae5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:49.373512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fad2a8-312a-4e30-bcb0-a7fc19c0ae5d service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:49.385946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1259/2531] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:49.456168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:49.476485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:49.522997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c2fa6-d6cd-405a-be43-cfedba07732a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:49.529933 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:49.530755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:49.531176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:49.531517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] PATH_INFO: `/v3/users/e76f0dbda87443cf8f2612c7208b98ff` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:49.532873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76f0dbda87443cf8f2612c7208b98ff)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:49.534898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:49.724986 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76f0dbda87443cf8f2612c7208b98ff'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:49.732275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76f0dbda87443cf8f2612c7208b98ff'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:49.732967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76f0dbda87443cf8f2612c7208b98ff'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:49.745155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76f0dbda87443cf8f2612c7208b98ff'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:49.825091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76f0dbda87443cf8f2612c7208b98ff'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:49.833270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1273/2532] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:49:49 2022] DELETE /identity/v3/users/e76f0dbda87443cf8f2612c7208b98ff => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:49.837991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:49.838502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:49.838502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:49.859497 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:49.860716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:49.921659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1260/2533] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:49.931422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:49.933002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:50.005270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f799c9-308c-4701-94dd-ad7984ce2c8e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:50.009698 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:50.010321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:50.010693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:50.010951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:50.021853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:50.023380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:50.033989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1274/2534] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:50.518114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:50.519718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:50.537165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:50.537939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309db8a2-d11a-4048-be97-1685f502c9c5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:50.556031 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:50.556701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:50.556989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:50.557451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] PATH_INFO: `/v3/projects/2bbd6f9f5bd54142b4acb1973b15862d` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:50.558551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bbd6f9f5bd54142b4acb1973b15862d)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:50.560492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:50.687726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bbd6f9f5bd54142b4acb1973b15862d', 'request_id': 'req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:50.695002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bbd6f9f5bd54142b4acb1973b15862d', 'request_id': 'req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:50.740947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bbd6f9f5bd54142b4acb1973b15862d', 'request_id': 'req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:50.754653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] The token cache is being invalidate because project 2bbd6f9f5bd54142b4acb1973b15862d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:49:50.754970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:50.755645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:50.757532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1261/2535] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:49:50 2022] DELETE /identity/v3/projects/2bbd6f9f5bd54142b4acb1973b15862d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:50.762896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:50.775781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a681a3-f562-4d15-a0f9-f9d2f2402138 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:50.819619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:50.846724 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:50.847372 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:50.848205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:50.848561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] PATH_INFO: `/v3/users/1b14e158c16b4c719fef005bcd1c5a6f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:50.849916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b14e158c16b4c719fef005bcd1c5a6f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:50.851931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:51.014116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b14e158c16b4c719fef005bcd1c5a6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:51.021746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b14e158c16b4c719fef005bcd1c5a6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:51.022529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b14e158c16b4c719fef005bcd1c5a6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:51.038684 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b14e158c16b4c719fef005bcd1c5a6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:51.088577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2e585f00-1c31-461f-99d2-41be394c435f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b14e158c16b4c719fef005bcd1c5a6f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:51.096270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1275/2536] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:49:50 2022] DELETE /identity/v3/users/1b14e158c16b4c719fef005bcd1c5a6f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:51.655515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:51.671980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:51.686744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3ac2a-1169-4332-b37f-d938adcf8ff9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:51.775917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:51.776581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:51.776849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:51.777125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] PATH_INFO: `/v3/projects/c91726a9ffc348128a4e9586c6cddfdb` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:51.778345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91726a9ffc348128a4e9586c6cddfdb)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:51.780261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:51.908115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91726a9ffc348128a4e9586c6cddfdb', 'request_id': 'req-24cb840b-9efb-437f-b232-ba517f7aa9f7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:51.915823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91726a9ffc348128a4e9586c6cddfdb', 'request_id': 'req-24cb840b-9efb-437f-b232-ba517f7aa9f7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:52.002149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91726a9ffc348128a4e9586c6cddfdb', 'request_id': 'req-24cb840b-9efb-437f-b232-ba517f7aa9f7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:52.014241 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] The token cache is being invalidate because project c91726a9ffc348128a4e9586c6cddfdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:49:52.014530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:52.015147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-24cb840b-9efb-437f-b232-ba517f7aa9f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:49:52.016665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1262/2537] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:49:51 2022] DELETE /identity/v3/projects/c91726a9ffc348128a4e9586c6cddfdb => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:52.022820 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.023348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.023455 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.043098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:52.044302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:52.111888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1276/2538] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:52.120186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.120486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.120771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.142689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:52.144187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:52.166389 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1263/2539] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:52.170067 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.180430 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b13b12-afc6-446f-a61c-c0b8fd6a0b63 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.184908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.185552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.185831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.186098 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.186789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:49:52.187090 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.188678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:52.201488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1277/2540] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:49:52.205714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.215153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9faa0c-2b88-4788-bc89-b48e9a5257c5 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.220515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.221316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.221613 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.221892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.222625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.224575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:52.234870 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:52.318944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1264/2541] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:49:52 2022] POST /identity/v3/projects => generated 407 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:52.323187 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.335833 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ac2df-0ffb-42a4-820e-730835f218d1 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.382505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:52.399445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.400952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.401236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.401507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.402185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.404204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:52.488934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1278/2542] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:49:52 2022] POST /identity/v3/users => generated 304 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:52.493262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.505306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba191e7-c85b-4f15-91e7-41046e24a5eb admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.510190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.510824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.512446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.512446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.512446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.513888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:52.524986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1265/2543] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:52.529079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.540233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119d9eb-b118-444b-9a5c-2fbefa2a9b42 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.545181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e14af-ff8b-4dab-af14-daf1d5533e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.545816 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e14af-ff8b-4dab-af14-daf1d5533e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.546096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e14af-ff8b-4dab-af14-daf1d5533e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.546371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e14af-ff8b-4dab-af14-daf1d5533e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.547077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e14af-ff8b-4dab-af14-daf1d5533e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.548690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e14af-ff8b-4dab-af14-daf1d5533e71 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:52.561642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1279/2544] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:49:52.565879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.576983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34171-170e-4211-aed7-b841c7f5cca0 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.581985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.582823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.583146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.583468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] PATH_INFO: `/v3/projects/880fce6b18bc4fd0ba9065ede0842b1a/users/41f63ef3ee3447e9a34316476b4a1b0a/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.599052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=880fce6b18bc4fd0ba9065ede0842b1a, user_id=41f63ef3ee3447e9a34316476b4a1b0a, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.601826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:52.715384 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1266/2545] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:52 2022] PUT /identity/v3/projects/880fce6b18bc4fd0ba9065ede0842b1a/users/41f63ef3ee3447e9a34316476b4a1b0a/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:49:52.734827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.735340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.735683 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.778478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] MFA Rules not processed for user `41f63ef3ee3447e9a34316476b4a1b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:52.779696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:52.857655 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1280/2546] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:52 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:52.862234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:52.959094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56932f30-aab2-49c3-a970-a782748f7d9e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:52.968178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:52.968178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:52.968178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:52.968178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:52.968757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:52.968882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.272548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1267/2547] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:49:52 2022] POST /identity/v3/projects => generated 409 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:53.276685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:53.287658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63283b-c4c4-4f81-81b1-3859dc1cc020 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:53.347669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:53.348211 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.348488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.348759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.349506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:53.351521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.444359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1281/2548] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:49:53 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:49:53.448172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:53.460416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871b7db-a2a1-416f-af91-27d964719a6d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:53.465674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:53.466299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.466583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.466854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.467667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:53.469370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.481193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1268/2549] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:49:53.485392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:53.496247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012d958-d642-43dc-ade8-e43af77604c5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:53.500708 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:53.501463 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.501744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.502015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] PATH_INFO: `/v3/projects/a0b4e8c330bf4a8f92bd3e4b2dffa161/users/a0d16ecc30a9475a873eaeae7d1f5958/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.520772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b4e8c330bf4a8f92bd3e4b2dffa161, user_id=a0d16ecc30a9475a873eaeae7d1f5958, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:53.524503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.602676 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1282/2550] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:53 2022] PUT /identity/v3/projects/a0b4e8c330bf4a8f92bd3e4b2dffa161/users/a0d16ecc30a9475a873eaeae7d1f5958/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:53.606789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:53.617775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc2d38-9085-4178-a00c-c4fcdca28a4f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:53.678350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5e65-de84-4544-b77a-173366fc20dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:53.678974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5e65-de84-4544-b77a-173366fc20dc admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.679257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5e65-de84-4544-b77a-173366fc20dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.679532 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5e65-de84-4544-b77a-173366fc20dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.680338 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d5e65-de84-4544-b77a-173366fc20dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:53.682152 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d5e65-de84-4544-b77a-173366fc20dc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.694254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1269/2551] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:49:53.698421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:53.710171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaa06f-04bf-466e-b045-f13db65b6203 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:53.715033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:53.716035 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.716410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.716776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] PATH_INFO: `/v3/projects/a0b4e8c330bf4a8f92bd3e4b2dffa161/users/a0d16ecc30a9475a873eaeae7d1f5958/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.718962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b4e8c330bf4a8f92bd3e4b2dffa161, user_id=a0d16ecc30a9475a873eaeae7d1f5958, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:53.721733 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.734018 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1283/2552] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:49:53 2022] PUT /identity/v3/projects/a0b4e8c330bf4a8f92bd3e4b2dffa161/users/a0d16ecc30a9475a873eaeae7d1f5958/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:49:53.747582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.748114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.748114 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.779565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] MFA Rules not processed for user `a0d16ecc30a9475a873eaeae7d1f5958`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:49:53.780800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:53.794710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:53.854702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1270/2553] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:49:53 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:49:53.866391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:53.868107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:49:53.875063 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:49:53.925436 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63b0f1-1a61-4cbe-9160-01b1a29fa007 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:53.930133 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:53.930738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:53.931014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:53.931278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:53.940232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:53.943713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:53.954435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1284/2554] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:53 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:49:54.278780 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:49:54.294601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d6bb3-86fe-4a25-9676-d750f6c25128 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:49:54.301304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c149bab-c45f-4efc-aeb9-0d1d6c0702f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:49:54.302265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c149bab-c45f-4efc-aeb9-0d1d6c0702f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:49:54.302551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c149bab-c45f-4efc-aeb9-0d1d6c0702f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:49:54.302852 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c149bab-c45f-4efc-aeb9-0d1d6c0702f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:49:54.375280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c149bab-c45f-4efc-aeb9-0d1d6c0702f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:49:54.376900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c149bab-c45f-4efc-aeb9-0d1d6c0702f1 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:49:54.389492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1271/2555] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:49:54 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:03.958674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=24 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:04.382348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=51 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:14.599853 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:14.662948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a77a8f-2d2c-47d7-a9f5-c9c94cd3398f service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:14.667439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:14.668062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:14.668388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:14.668661 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] PATH_INFO: `/v3/users/41f63ef3ee3447e9a34316476b4a1b0a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:14.669831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f63ef3ee3447e9a34316476b4a1b0a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:14.671756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:14.855762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f63ef3ee3447e9a34316476b4a1b0a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:14.862567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f63ef3ee3447e9a34316476b4a1b0a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:14.863284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f63ef3ee3447e9a34316476b4a1b0a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:14.875630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f63ef3ee3447e9a34316476b4a1b0a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:14.932363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f63ef3ee3447e9a34316476b4a1b0a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:14.938855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1285/2556] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:50:14 2022] DELETE /identity/v3/users/41f63ef3ee3447e9a34316476b4a1b0a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:14.943561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:14.944048 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:14.944192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:14.963104 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:14.964309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.022755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1272/2557] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:15.032271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:15.033719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.091737 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5763-db71-4222-b0b9-8cebd3d933dc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:15.095359 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:15.095970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:15.096248 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:15.096517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:15.106472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:15.107908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:15.118028 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1286/2558] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:15.564951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:15.566799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.583088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55372a07-b85c-4ad2-a20f-d23216a3f9fc None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:15.597866 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:15.600997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:15.600997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:15.600997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] PATH_INFO: `/v3/projects/880fce6b18bc4fd0ba9065ede0842b1a` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:15.600997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880fce6b18bc4fd0ba9065ede0842b1a)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:15.603273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:15.710081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880fce6b18bc4fd0ba9065ede0842b1a', 'request_id': 'req-b9126998-b388-4eea-a63a-976eb1265996'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:15.723306 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880fce6b18bc4fd0ba9065ede0842b1a', 'request_id': 'req-b9126998-b388-4eea-a63a-976eb1265996'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:15.735330 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:15.736903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.763601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:15.800934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880fce6b18bc4fd0ba9065ede0842b1a', 'request_id': 'req-b9126998-b388-4eea-a63a-976eb1265996'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:15.810006 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854b7c1-9229-4e90-9de5-f2b266a700f1 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:15.812828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] The token cache is being invalidate because project 880fce6b18bc4fd0ba9065ede0842b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:50:15.812828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:15.813185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:15.814805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1273/2559] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:50:15 2022] DELETE /identity/v3/projects/880fce6b18bc4fd0ba9065ede0842b1a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:15.818436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:15.818436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.882277 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9126998-b388-4eea-a63a-976eb1265996 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:15.886203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:15.886861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:15.887197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:15.887601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:15.889496 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.890666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:15.891440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:15.891730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:15.892001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:15.893789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.906827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:15.908497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:15.919770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1287/2560] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:50:15.921100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:15.922424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:15.933481 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:15.963810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:15.965398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:15.976968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1274/2561] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:16.000909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9243e-099b-40b3-a14b-fc71fd1491c3 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:16.005896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:16.006688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:16.007065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:16.007272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] PATH_INFO: `/v3/users/a0d16ecc30a9475a873eaeae7d1f5958` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:16.008450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d16ecc30a9475a873eaeae7d1f5958)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:16.010468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:16.216850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d16ecc30a9475a873eaeae7d1f5958'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:16.218283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:16.226019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d16ecc30a9475a873eaeae7d1f5958'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:16.227158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d16ecc30a9475a873eaeae7d1f5958'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:16.241188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d16ecc30a9475a873eaeae7d1f5958'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:16.332923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-178f4cc3-54f9-49b1-bead-96612508c992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d16ecc30a9475a873eaeae7d1f5958'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:16.340888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1288/2562] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:50:15 2022] DELETE /identity/v3/users/a0d16ecc30a9475a873eaeae7d1f5958 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:16.842618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:16.853987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7127a-a22b-4bc3-8e9b-ecf1bef31553 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:16.918095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:16.918785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:16.919072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:16.919377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] PATH_INFO: `/v3/projects/a0b4e8c330bf4a8f92bd3e4b2dffa161` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:16.920524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b4e8c330bf4a8f92bd3e4b2dffa161)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:16.922534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.036184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b4e8c330bf4a8f92bd3e4b2dffa161', 'request_id': 'req-f83b6998-5235-4d2f-91d7-b3853e3d439c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:17.043343 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b4e8c330bf4a8f92bd3e4b2dffa161', 'request_id': 'req-f83b6998-5235-4d2f-91d7-b3853e3d439c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:17.116234 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b4e8c330bf4a8f92bd3e4b2dffa161', 'request_id': 'req-f83b6998-5235-4d2f-91d7-b3853e3d439c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:17.127281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] The token cache is being invalidate because project a0b4e8c330bf4a8f92bd3e4b2dffa161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:50:17.127575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:17.128207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f83b6998-5235-4d2f-91d7-b3853e3d439c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:17.129740 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1275/2563] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:50:16 2022] DELETE /identity/v3/projects/a0b4e8c330bf4a8f92bd3e4b2dffa161 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:17.135804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.136297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.136422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.156354 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:17.157612 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:17.234849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1289/2564] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:17.243604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.243892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.244286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.265382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:17.266581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:17.282331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1276/2565] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:17.283084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.293618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc28bf-07d8-4900-b4df-cf5783b25012 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:17.298650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:17.299270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.299547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.299815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.301416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:50:17.301767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:17.303630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.315740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1290/2566] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:50:17.320786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.330645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c912521-3527-4fd2-a492-551067b84fcf None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:17.338247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:17.338864 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.339176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.339445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.340108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:17.342062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.426861 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1277/2567] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:50:17 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:50:17.433729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.446136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53335dd0-0ebd-4e48-9057-98b48c60ad51 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:17.507279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:17.508001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.508286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.508559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.509275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:17.511209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.619914 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1291/2568] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:50:17 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:50:17.624142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.635831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958cba6-b324-4529-93ce-84c2e39df6b6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:17.640697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:17.641348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.641686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.641961 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.642661 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:17.644278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.654931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1278/2569] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:50:17.658867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.669318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506c65f-5cee-4a1a-b17e-c10fdca52106 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:17.674300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6eabf-0db3-4433-8cbb-709bcfb67481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:17.675427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6eabf-0db3-4433-8cbb-709bcfb67481 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.675427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6eabf-0db3-4433-8cbb-709bcfb67481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.675427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6eabf-0db3-4433-8cbb-709bcfb67481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.676112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6eabf-0db3-4433-8cbb-709bcfb67481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:17.678318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6eabf-0db3-4433-8cbb-709bcfb67481 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.690052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1292/2570] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:50:17.694778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.704745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598efd7-711a-438e-bb43-290310697517 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:17.709414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:17.710077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.710362 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.710666 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] PATH_INFO: `/v3/projects/1dd657bfb46942d798e6972c64706a4e/users/48d105f34f6d42fc800517b33a30d2c8/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.726429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd657bfb46942d798e6972c64706a4e, user_id=48d105f34f6d42fc800517b33a30d2c8, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:17.729484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:17.804428 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:17.809762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1279/2571] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:50:17 2022] PUT /identity/v3/projects/1dd657bfb46942d798e6972c64706a4e/users/48d105f34f6d42fc800517b33a30d2c8/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:17.821780 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207d82-5c9b-4bb1-b807-d96150f2e001 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:17.822348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207d82-5c9b-4bb1-b807-d96150f2e001 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:17.823552 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207d82-5c9b-4bb1-b807-d96150f2e001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:17.860945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-44207d82-5c9b-4bb1-b807-d96150f2e001 None None] MFA Rules not processed for user `48d105f34f6d42fc800517b33a30d2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:17.863389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-44207d82-5c9b-4bb1-b807-d96150f2e001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:17.887643 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:17.954467 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1293/2572] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:17.955727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:17.957922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:18.031526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fe19e-4d81-41e0-8054-44640bbc8398 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:18.035309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:18.035927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:18.037649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:18.037649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:18.046110 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:18.056770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:18.057636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1280/2573] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:18.482577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca1dee-2b56-442d-bcae-4bb50174bb28 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:18.483103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca1dee-2b56-442d-bcae-4bb50174bb28 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:18.483103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca1dee-2b56-442d-bcae-4bb50174bb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:18.503759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f8ca1dee-2b56-442d-bcae-4bb50174bb28 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:18.505094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f8ca1dee-2b56-442d-bcae-4bb50174bb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:18.579356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1294/2574] 10.222.0.66 () {58 vars in 967 bytes} [Thu Aug 4 03:50:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:18.602914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:18.604655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:18.711043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e71e3b-2738-409f-9066-c628a8c0ba04 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:18.716682 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:18.717355 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:18.717643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:18.717921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:18.733091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:18.738785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:18.744943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1281/2575] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:50:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:50:19.785045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd2fc8-ded4-4f99-8f8f-fdcf43abbce0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:19.788315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd2fc8-ded4-4f99-8f8f-fdcf43abbce0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:19.788315 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd2fc8-ded4-4f99-8f8f-fdcf43abbce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:19.805380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0ffd2fc8-ded4-4f99-8f8f-fdcf43abbce0 None None] MFA Rules not processed for user `48d105f34f6d42fc800517b33a30d2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:19.806629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0ffd2fc8-ded4-4f99-8f8f-fdcf43abbce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:19.813711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1295/2576] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:20.332616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:20.334482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:20.413663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02309eef-47ac-4981-98be-8aadef192ba0 service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:20.418727 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:20.419349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:20.419640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:20.419918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:20.421703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:20.432995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:20.434643 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:20.444544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1282/2577] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:29.799843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=46 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:30.440348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=90 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:38.502760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46447a-600d-4396-acf0-cf57209d2a09 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:38.503238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46447a-600d-4396-acf0-cf57209d2a09 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:38.503345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46447a-600d-4396-acf0-cf57209d2a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:38.523215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-3d46447a-600d-4396-acf0-cf57209d2a09 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:38.524422 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3d46447a-600d-4396-acf0-cf57209d2a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:38.531589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1296/2578] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:50:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:50:38.543400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:38.555446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bef4a-9ccf-49ba-9024-8aea3e50348c service placement] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:38.560494 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:38.563012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:38.563012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:38.563012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:38.570709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:38.572328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:38.580374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1283/2579] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:50:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:50:39.488250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ddcbd-5e48-428d-abc3-58c30b2a5074 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:39.488538 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ddcbd-5e48-428d-abc3-58c30b2a5074 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:39.488844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ddcbd-5e48-428d-abc3-58c30b2a5074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:39.508520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-ee9ddcbd-5e48-428d-abc3-58c30b2a5074 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:39.509754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ee9ddcbd-5e48-428d-abc3-58c30b2a5074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:39.519821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1297/2580] 10.222.0.66 () {58 vars in 967 bytes} [Thu Aug 4 03:50:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:39.532567 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:39.543214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a796-e9de-43c3-a509-00fe6e3e297f service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:39.548597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:39.549236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:39.549535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:39.549816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:39.558077 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:39.559747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:39.567805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1284/2581] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:50:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:50:40.108393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067dc7d-2c0f-4918-ada9-6aec0e4c035a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:40.108674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067dc7d-2c0f-4918-ada9-6aec0e4c035a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:40.108954 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067dc7d-2c0f-4918-ada9-6aec0e4c035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:40.127858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-c067dc7d-2c0f-4918-ada9-6aec0e4c035a None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:40.129040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-c067dc7d-2c0f-4918-ada9-6aec0e4c035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:40.136171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1298/2582] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Aug 4 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:40.148379 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:40.158860 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb5f22-09fc-44ed-95ba-525983c694d9 service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:40.164747 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:40.165393 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:40.165672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:40.165940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:40.176546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:40.178506 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:40.187536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1285/2583] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:50:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 03:50:40.730404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:40.730865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:40.730973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:40.749981 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] MFA Rules not processed for user `4444ac94fc184e949c285d365931d0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:40.751171 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:40.759062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1299/2584] 10.222.0.66 () {58 vars in 967 bytes} [Thu Aug 4 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:40.770967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:40.781641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda224-b1e8-45c8-b861-415577de1f36 service swift] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:40.786612 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41e52-c32f-4a0f-ae7b-37e3dc30da9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd647c7fcd5fb490db5302a9ead7ab330', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:40.791914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41e52-c32f-4a0f-ae7b-37e3dc30da9c service swift] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:40.791914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41e52-c32f-4a0f-ae7b-37e3dc30da9c service swift] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:40.791914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41e52-c32f-4a0f-ae7b-37e3dc30da9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:40.796044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41e52-c32f-4a0f-ae7b-37e3dc30da9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:40.797665 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41e52-c32f-4a0f-ae7b-37e3dc30da9c service swift] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:40.806554 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1286/2585] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Aug 4 03:50:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 03:50:43.501241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:43.566956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e448b5d-1386-45d0-bf9f-679c193eef01 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:43.571503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:43.572144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:43.572405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:43.572670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] PATH_INFO: `/v3/users/48d105f34f6d42fc800517b33a30d2c8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:43.573844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d105f34f6d42fc800517b33a30d2c8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:43.575761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:43.704542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d105f34f6d42fc800517b33a30d2c8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:43.711821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d105f34f6d42fc800517b33a30d2c8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:43.712484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d105f34f6d42fc800517b33a30d2c8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:43.723988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d105f34f6d42fc800517b33a30d2c8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:43.789490 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d105f34f6d42fc800517b33a30d2c8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:43.795746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1300/2586] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:50:43 2022] DELETE /identity/v3/users/48d105f34f6d42fc800517b33a30d2c8 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:43.800307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:43.800787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:43.800898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:43.819607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:43.820814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:43.879082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1287/2587] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:43.889487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:43.890893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:43.947491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69216f8d-5940-4bdb-b889-ac75991073d3 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:43.951215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7f4ed-c81b-47de-a765-0099f9abf85e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:43.951832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7f4ed-c81b-47de-a765-0099f9abf85e service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:43.952110 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7f4ed-c81b-47de-a765-0099f9abf85e service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:43.952384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7f4ed-c81b-47de-a765-0099f9abf85e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:43.959189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:43.961541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7f4ed-c81b-47de-a765-0099f9abf85e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:43.962974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7f4ed-c81b-47de-a765-0099f9abf85e service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:43.972191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1301/2588] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:44.197633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.199100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:44.211245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e0c95-98a2-4449-ace4-4b1f6da4eab9 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.215892 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.216540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.216825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.217098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] PATH_INFO: `/v3/projects/1dd657bfb46942d798e6972c64706a4e` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.218293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd657bfb46942d798e6972c64706a4e)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:44.222963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:44.336290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd657bfb46942d798e6972c64706a4e', 'request_id': 'req-c2e10751-0a82-412f-bfc6-8d725b1e7a01'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:44.342623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd657bfb46942d798e6972c64706a4e', 'request_id': 'req-c2e10751-0a82-412f-bfc6-8d725b1e7a01'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:44.429206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd657bfb46942d798e6972c64706a4e', 'request_id': 'req-c2e10751-0a82-412f-bfc6-8d725b1e7a01'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:44.440205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] The token cache is being invalidate because project 1dd657bfb46942d798e6972c64706a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:50:44.440492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:44.441270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c2e10751-0a82-412f-bfc6-8d725b1e7a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:50:44.442814 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1288/2589] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:50:44 2022] DELETE /identity/v3/projects/1dd657bfb46942d798e6972c64706a4e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:44.447762 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.448054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.448332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.468345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:44.469582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:44.529059 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1302/2590] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:44.537502 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.538142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.538142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.557846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:44.559038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:44.566800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1289/2591] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:44.571570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.581451 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a0001-6b6c-4ef5-84c3-27270444e9a9 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.586227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.587709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.587709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.587709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.588176 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:50:44.588363 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:44.590021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:44.601380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1303/2592] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:50:44.605913 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.615233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ad635-f1a0-49fa-b171-d185a32852ba None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.621060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.621701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.621987 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.622264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.622939 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:44.624862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:44.704354 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1290/2593] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:50:44 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:50:44.708577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.718969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720c2b8-07a1-4a35-a096-f9c23f2d05cc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.780146 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.780754 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.781036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.781349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.782020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:44.783948 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:44.897976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1304/2594] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:50:44 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:50:44.902126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.907623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:44.913589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d7b6e-6645-40e1-a489-c1b252a703c2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.918339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.918955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.919240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.919517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.920213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:44.921934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:44.932583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1291/2595] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:50:44.936448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.946795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e34d8-59ac-48da-8929-eb2ad67c8ebd admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.951628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01951c12-bf20-4879-a64c-ff0ff49ccb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.952237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01951c12-bf20-4879-a64c-ff0ff49ccb74 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.954722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01951c12-bf20-4879-a64c-ff0ff49ccb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.954722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01951c12-bf20-4879-a64c-ff0ff49ccb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:44.954722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01951c12-bf20-4879-a64c-ff0ff49ccb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:44.955312 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01951c12-bf20-4879-a64c-ff0ff49ccb74 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:44.966822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1305/2596] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:50:44.970828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:44.982220 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf5d57-e749-406c-823e-73643258b54e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:44.986940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:44.987664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:44.988207 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:44.988719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] PATH_INFO: `/v3/projects/70809dff6be74d999cc2c7af5cd62c2b/users/8bbeb376b3f34bdc8a9e3acd498f56b3/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:45.005873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70809dff6be74d999cc2c7af5cd62c2b, user_id=8bbeb376b3f34bdc8a9e3acd498f56b3, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:45.008817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:45.063120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1292/2597] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:50:44 2022] PUT /identity/v3/projects/70809dff6be74d999cc2c7af5cd62c2b/users/8bbeb376b3f34bdc8a9e3acd498f56b3/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:50:45.073456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:45.073742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:45.074014 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:45.101763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] MFA Rules not processed for user `8bbeb376b3f34bdc8a9e3acd498f56b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:50:45.103031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:45.150425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:45.167777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1306/2598] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:50:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:50:45.178719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:50:45.180624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:50:45.237633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0459-4aa5-4732-9b8f-03d21b22acc7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:50:45.241378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334319e7-f1ea-4ca3-a41e-021756e48cc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:50:45.241995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334319e7-f1ea-4ca3-a41e-021756e48cc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:50:45.242282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334319e7-f1ea-4ca3-a41e-021756e48cc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:50:45.242562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334319e7-f1ea-4ca3-a41e-021756e48cc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:50:45.250735 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334319e7-f1ea-4ca3-a41e-021756e48cc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:50:45.252179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334319e7-f1ea-4ca3-a41e-021756e48cc1 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:50:45.260993 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1293/2599] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:50:45 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:50:55.193398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:50:55.256965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:10.140409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:10.219407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868b319-5bfe-4960-afe6-dda47131353e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:10.224017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:10.224652 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:10.224930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:10.225227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] PATH_INFO: `/v3/users/8bbeb376b3f34bdc8a9e3acd498f56b3` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:10.226585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbeb376b3f34bdc8a9e3acd498f56b3)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:10.231600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:10.407976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbeb376b3f34bdc8a9e3acd498f56b3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:10.415713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbeb376b3f34bdc8a9e3acd498f56b3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:10.416842 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbeb376b3f34bdc8a9e3acd498f56b3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:10.428295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbeb376b3f34bdc8a9e3acd498f56b3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:10.480042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbeb376b3f34bdc8a9e3acd498f56b3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:10.486426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1307/2600] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:51:10 2022] DELETE /identity/v3/users/8bbeb376b3f34bdc8a9e3acd498f56b3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:10.491137 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:10.496418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:10.496418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:10.511510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:51:10.512750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:10.574769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1294/2601] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:51:10.585518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:10.587999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:10.645115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242a70e-1408-4719-a6f4-02e274cdf311 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:10.648773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:10.650251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:10.650251 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:10.650500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:10.665588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:10.665588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:10.672599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1308/2602] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:11.202447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:11.203869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:11.217165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d6e03-5db1-4531-900c-37609ff977dd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:11.222002 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:11.222720 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:11.223066 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:11.223427 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] PATH_INFO: `/v3/projects/70809dff6be74d999cc2c7af5cd62c2b` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:11.224652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70809dff6be74d999cc2c7af5cd62c2b)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:11.226601 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:11.326141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70809dff6be74d999cc2c7af5cd62c2b', 'request_id': 'req-72330a0c-12b2-41bd-9cef-c72792a1f6c0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:11.332225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70809dff6be74d999cc2c7af5cd62c2b', 'request_id': 'req-72330a0c-12b2-41bd-9cef-c72792a1f6c0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:11.401275 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70809dff6be74d999cc2c7af5cd62c2b', 'request_id': 'req-72330a0c-12b2-41bd-9cef-c72792a1f6c0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:11.412294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] The token cache is being invalidate because project 70809dff6be74d999cc2c7af5cd62c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:51:11.412580 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:11.413225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-72330a0c-12b2-41bd-9cef-c72792a1f6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:11.415522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1295/2603] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:51:11 2022] DELETE /identity/v3/projects/70809dff6be74d999cc2c7af5cd62c2b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:20.668184 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:21.408956 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:22.065265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:22.066728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:22.161945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b812a9-5174-44e7-b858-1f3b64512487 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:22.381759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:22.382371 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:22.382651 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:22.382930 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:22.385263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:22.397588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:22.399149 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:22.411566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1309/2604] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:26.920164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:26.920667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:26.920768 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:26.940178 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] MFA Rules not processed for user `ac99f60384fe4a119d019cedfe1d3557`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:51:26.941538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:26.997628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1296/2605] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:51:27.303022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:27.304547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:27.359500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a310-6c32-463a-ab4c-95830b2a4343 service nova] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:27.363036 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:27.363636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:27.363913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:27.364181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:27.365892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:27.375062 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:27.376475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:27.384203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1310/2606] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:36.990280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=25 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:37.380766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=88 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:39.425788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:39.427111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:39.442948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efd62f-9236-467e-b625-4fe645adba0b None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:39.448544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:39.449143 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:39.449452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:39.449725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:39.451411 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:39.461487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:39.463063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:39.472505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1297/2607] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:47.063592 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:47.064975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:47.121858 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b183e6e-e5fe-4c52-b967-0dc422a0c3ec service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:47.125415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:47.126021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:47.126299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:47.126573 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:47.128332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:47.207365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:47.208824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:47.226987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1311/2608] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:48.576149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:48.585478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a412d2d-ef3c-477f-8035-f73ebb795717 service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:48.589817 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:48.590422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:48.590700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:48.590972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:48.592703 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:48.654900 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:48.656339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:48.667718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1298/2609] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:52.342666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:52.345181 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:52.416230 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec428d12-936a-4844-bdbb-6a896b66e779 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:52.421397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:52.422001 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:52.422282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:52.422557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:52.424550 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:52.437097 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:52.438777 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:52.449717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1312/2610] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:51:52.468044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:52.469583 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:52.484233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365dfac-d3a1-423f-aea2-b9106a5eda38 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:52.489560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:52.490208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:52.490519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:52.490798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] PATH_INFO: `/v3/users/55beb38ea2994ea7987fe546235b58a5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:52.491962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55beb38ea2994ea7987fe546235b58a5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:52.494044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:52.681563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55beb38ea2994ea7987fe546235b58a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:52.682793 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55beb38ea2994ea7987fe546235b58a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:52.683359 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55beb38ea2994ea7987fe546235b58a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:52.695120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55beb38ea2994ea7987fe546235b58a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:52.770560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-4ab57730-b5da-40f5-9374-952d8f2b2b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55beb38ea2994ea7987fe546235b58a5'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:52.783134 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1299/2611] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:51:52 2022] DELETE /identity/v3/users/55beb38ea2994ea7987fe546235b58a5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:53.001101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.012512 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ebfb-5dce-4064-9748-47a1abcbdb27 service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.025270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=100 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:53.071841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.072477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.073934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.073934 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] PATH_INFO: `/v3/projects/274bea7582354f49841b893c0a39ffdf` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.075247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=274bea7582354f49841b893c0a39ffdf)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.077225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.159424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274bea7582354f49841b893c0a39ffdf', 'request_id': 'req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:53.166025 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274bea7582354f49841b893c0a39ffdf', 'request_id': 'req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:53.238055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274bea7582354f49841b893c0a39ffdf', 'request_id': 'req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:53.248807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] The token cache is being invalidate because project 274bea7582354f49841b893c0a39ffdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:51:53.249091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:53.249735 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ca0ed603-0e32-4a7f-a0ab-572e27ea0a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:51:53.251232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1313/2612] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:51:53 2022] DELETE /identity/v3/projects/274bea7582354f49841b893c0a39ffdf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:53.256269 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.257032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.257032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.277131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:51:53.278366 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:53.325950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:53.337824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1300/2613] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:51:53.347454 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.347917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.348015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.367476 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:51:53.368665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:53.376515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1314/2614] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:51:53.380693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.391082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1727d0-063f-4455-befb-49e541ea8e31 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.395655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.396263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.396545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.396819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.397568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:51:53.397899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.399501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.411522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1301/2615] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:51:53.415893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.425619 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af69e-b419-4f7d-97e8-1f0a6edeb1ae None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.430398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.430992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.431267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.431532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.432711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.434689 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.520437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1315/2616] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:51:53 2022] POST /identity/v3/projects => generated 357 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:51:53.524536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.535375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406044ff-6ba8-4bd9-9279-41b42234e1e5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.593420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.594041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.594357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.594636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.595313 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.597328 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.696294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1302/2617] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:51:53 2022] POST /identity/v3/users => generated 279 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:51:53.700811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.711054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed567a49-abce-4457-8323-13932e76898e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.715577 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.716177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.716456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.716726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.717453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.719043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.729156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1316/2618] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:51:53.733014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.742863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763af0c-a68a-43e9-b7a9-9fdf958bde9a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.748015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e37164-c391-4588-a886-268c430b31b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.748610 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e37164-c391-4588-a886-268c430b31b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.748893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e37164-c391-4588-a886-268c430b31b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.749166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e37164-c391-4588-a886-268c430b31b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.749891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e37164-c391-4588-a886-268c430b31b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.751485 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e37164-c391-4588-a886-268c430b31b4 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.763708 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1303/2619] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:51:53.768385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:53.778304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529da0-0e7e-4497-8ea2-486e3bd88f8a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:53.783385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:53.784033 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.784300 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.784566 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] PATH_INFO: `/v3/projects/d9e4f31fa40543f8ae5b8e0d948bf500/users/ec094ec920c34ab5a92187b991a60ef6/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.799533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e4f31fa40543f8ae5b8e0d948bf500, user_id=ec094ec920c34ab5a92187b991a60ef6, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:53.802199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:53.901851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1317/2620] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:51:53 2022] PUT /identity/v3/projects/d9e4f31fa40543f8ae5b8e0d948bf500/users/ec094ec920c34ab5a92187b991a60ef6/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:53.912307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:53.912605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:53.914726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:53.940765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] MFA Rules not processed for user `ec094ec920c34ab5a92187b991a60ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:51:53.942025 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:54.005938 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1304/2621] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:51:54.010964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.054308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:54.071920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fd449-4cd4-46d9-9cbc-47ddb9f3e896 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.076704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.077407 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.077686 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.077955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.078637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.080568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.190334 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1318/2622] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:51:54 2022] POST /identity/v3/projects => generated 357 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:51:54.196284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.207487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5409dd-14bd-4542-bcc2-95885137cc8a None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.251790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:54.263574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.264199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.264482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.264758 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.265461 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.267490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.330603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1305/2623] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:51:54 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:51:54.334654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.345116 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad2923-12da-4947-9360-34985432b533 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.350147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.350749 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.351022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.351290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.351975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.354461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.364793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1319/2624] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:51:54.368953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.379646 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3043013-453c-4077-9ca7-2969e1f629d2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.384239 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.384895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.385349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.385609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] PATH_INFO: `/v3/projects/a07f3dfed0294fa59288afb2f47f700a/users/87a5a942e4f84195ba1451385deb3282/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.400420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07f3dfed0294fa59288afb2f47f700a, user_id=87a5a942e4f84195ba1451385deb3282, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.403689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.485377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1306/2625] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:51:54 2022] PUT /identity/v3/projects/a07f3dfed0294fa59288afb2f47f700a/users/87a5a942e4f84195ba1451385deb3282/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:54.493293 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.503158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd882a9-b1a0-4e67-896e-aef8cce152a9 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.559188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51239145-6182-46cf-85af-64865c9a0db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.559808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51239145-6182-46cf-85af-64865c9a0db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.560088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51239145-6182-46cf-85af-64865c9a0db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.560357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51239145-6182-46cf-85af-64865c9a0db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.561044 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51239145-6182-46cf-85af-64865c9a0db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.562669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51239145-6182-46cf-85af-64865c9a0db4 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.572321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1320/2626] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:51:54.576799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.588731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca78fd0-a83e-4fb0-a253-6650a4afc83e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.593091 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.593785 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.594067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.594342 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] PATH_INFO: `/v3/projects/a07f3dfed0294fa59288afb2f47f700a/users/87a5a942e4f84195ba1451385deb3282/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.596311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07f3dfed0294fa59288afb2f47f700a, user_id=87a5a942e4f84195ba1451385deb3282, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.599546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.610357 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1307/2627] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:51:54 2022] PUT /identity/v3/projects/a07f3dfed0294fa59288afb2f47f700a/users/87a5a942e4f84195ba1451385deb3282/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:51:54.627405 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.627701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.627997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.655298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] MFA Rules not processed for user `87a5a942e4f84195ba1451385deb3282`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:51:54.656497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:54.721747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1321/2628] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:51:54.735405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:54.737086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:51:54.794408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ea143-bb1c-43a3-90d7-281d7d171685 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:54.798135 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:54.798746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:54.799028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:54.799333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:54.808281 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:54.809790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:54.819096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1308/2629] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:51:55.010510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:51:55.020013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336cd6b8-7146-4417-baa2-84f8cfd1a108 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:51:55.023517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bcff7-5914-4f5e-9f1c-6e94df24ac07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:51:55.024114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bcff7-5914-4f5e-9f1c-6e94df24ac07 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:51:55.024387 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bcff7-5914-4f5e-9f1c-6e94df24ac07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:51:55.024653 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bcff7-5914-4f5e-9f1c-6e94df24ac07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:51:55.058337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:51:55.081460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bcff7-5914-4f5e-9f1c-6e94df24ac07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:51:55.082901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bcff7-5914-4f5e-9f1c-6e94df24ac07 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:51:55.093047 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1322/2630] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:51:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:04.815008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:05.118973 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=11 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:12.254227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:12.316523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e24e86-aef1-42da-b324-0b049ac29bea service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:12.321012 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:12.321687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:12.321974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:12.322253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] PATH_INFO: `/v3/users/ec094ec920c34ab5a92187b991a60ef6` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:12.323419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec094ec920c34ab5a92187b991a60ef6)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:12.325382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:12.543588 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec094ec920c34ab5a92187b991a60ef6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:12.544253 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec094ec920c34ab5a92187b991a60ef6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:12.544855 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec094ec920c34ab5a92187b991a60ef6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:12.556784 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec094ec920c34ab5a92187b991a60ef6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:12.627975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec094ec920c34ab5a92187b991a60ef6'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:12.640365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1309/2631] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:12 2022] DELETE /identity/v3/users/ec094ec920c34ab5a92187b991a60ef6 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:12.644903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:12.645418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:12.645520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:12.664152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:12.665373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:12.723309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1323/2632] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:12.733488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:12.734883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:12.791679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170201d-3999-45d3-9d91-da0b984dac83 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:12.795278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:12.795887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:12.796172 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:12.796450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:12.806315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:12.807759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:12.817724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1310/2633] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:13.195669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:13.197289 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:13.209921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be503f8b-12a5-43bd-ad9d-97634939fa69 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:13.214555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:13.215988 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:13.216464 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:13.216748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] PATH_INFO: `/v3/projects/d9e4f31fa40543f8ae5b8e0d948bf500` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:13.217925 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e4f31fa40543f8ae5b8e0d948bf500)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:13.220531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:13.303124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e4f31fa40543f8ae5b8e0d948bf500', 'request_id': 'req-2ae045ef-7f50-4181-bf16-161a8e5b1f08'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.309924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e4f31fa40543f8ae5b8e0d948bf500', 'request_id': 'req-2ae045ef-7f50-4181-bf16-161a8e5b1f08'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.358849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e4f31fa40543f8ae5b8e0d948bf500', 'request_id': 'req-2ae045ef-7f50-4181-bf16-161a8e5b1f08'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.370337 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] The token cache is being invalidate because project d9e4f31fa40543f8ae5b8e0d948bf500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:13.378398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.378398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.378398 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1324/2634] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:13 2022] DELETE /identity/v3/projects/d9e4f31fa40543f8ae5b8e0d948bf500 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:13.378730 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:13.386787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130cee1-a198-4afe-bac5-568fa7127c15 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:13.404942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:13.441657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:13.442266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:13.442545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:13.442816 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] PATH_INFO: `/v3/users/87a5a942e4f84195ba1451385deb3282` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:13.443951 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a5a942e4f84195ba1451385deb3282)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:13.445916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:13.639835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a5a942e4f84195ba1451385deb3282'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.640546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a5a942e4f84195ba1451385deb3282'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.641186 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a5a942e4f84195ba1451385deb3282'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.653319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a5a942e4f84195ba1451385deb3282'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.741691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-35b49e73-0d51-4081-a2bf-36002fba7e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a5a942e4f84195ba1451385deb3282'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:13.754774 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1311/2635] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:13 2022] DELETE /identity/v3/users/87a5a942e4f84195ba1451385deb3282 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:14.098443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:14.099937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:14.115152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae045ef-7f50-4181-bf16-161a8e5b1f08 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:14.171560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:14.172194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.172471 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.172738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] PATH_INFO: `/v3/projects/a07f3dfed0294fa59288afb2f47f700a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.174571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07f3dfed0294fa59288afb2f47f700a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:14.176295 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:14.291920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:14.307593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07f3dfed0294fa59288afb2f47f700a', 'request_id': 'req-55534462-74d1-468c-bb60-c8bbc2766038'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:14.314224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07f3dfed0294fa59288afb2f47f700a', 'request_id': 'req-55534462-74d1-468c-bb60-c8bbc2766038'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:14.438830 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07f3dfed0294fa59288afb2f47f700a', 'request_id': 'req-55534462-74d1-468c-bb60-c8bbc2766038'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:14.450096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] The token cache is being invalidate because project a07f3dfed0294fa59288afb2f47f700a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:14.450502 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:14.451152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-55534462-74d1-468c-bb60-c8bbc2766038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:14.452696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1325/2636] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:14 2022] DELETE /identity/v3/projects/a07f3dfed0294fa59288afb2f47f700a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:14.458296 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.458800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.458898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.480022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:14.481260 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:14.540155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1312/2637] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:14.548101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.548924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.549232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.568859 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:14.570094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:14.578274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1326/2638] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:14.582965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:14.592674 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b3c9f-e3a5-4f4a-8cd5-cd990a3db0fd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:14.597332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:14.597936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.598216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.598487 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.599225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:52:14.599512 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:14.601111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:14.612902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1313/2639] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:52:14.617021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:14.626611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838ae3c-b701-4041-bb0c-b0c96a110ffe None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:14.631755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:14.632352 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.632628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.632893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.633586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:14.635468 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:14.798931 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1327/2640] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:52:14 2022] POST /identity/v3/projects => generated 363 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:14.803016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:14.813009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695ff41-0a6e-4f9b-b695-e6557b87176c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:14.839648 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:14.869672 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:14.870301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.870595 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.871229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.872228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:14.874365 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:14.963332 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1314/2641] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:52:14 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:14.967247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:14.978765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828a40-2885-45c3-98fb-cb3fd450035b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:14.983448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:14.984046 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:14.984326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:14.984593 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:14.985309 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:14.989865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:14.997118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1328/2642] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:15.000940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.011598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39529251-3f14-4809-9125-4037a9bdcf79 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.016227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933b323-a362-44b0-8f01-00d03a0a3869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.017614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b323-a362-44b0-8f01-00d03a0a3869 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.017903 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b323-a362-44b0-8f01-00d03a0a3869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.018181 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933b323-a362-44b0-8f01-00d03a0a3869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.018876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b323-a362-44b0-8f01-00d03a0a3869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.020493 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933b323-a362-44b0-8f01-00d03a0a3869 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.031016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1315/2643] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:15.035775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.045756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586086d-6e81-4e5f-954f-db708782bda8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.050361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.050984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.051263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.051537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] PATH_INFO: `/v3/projects/bbe237eca964481396a599237889cc7a/users/4ad0734fdebe48a2b21357857777728d/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.065987 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe237eca964481396a599237889cc7a, user_id=4ad0734fdebe48a2b21357857777728d, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.068637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.134644 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1329/2644] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:15 2022] PUT /identity/v3/projects/bbe237eca964481396a599237889cc7a/users/4ad0734fdebe48a2b21357857777728d/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:15.145641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63209-509c-425d-9076-72054f8d5520 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.145935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63209-509c-425d-9076-72054f8d5520 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.146214 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d63209-509c-425d-9076-72054f8d5520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.176023 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-02d63209-509c-425d-9076-72054f8d5520 None None] MFA Rules not processed for user `4ad0734fdebe48a2b21357857777728d`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:15.177265 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-02d63209-509c-425d-9076-72054f8d5520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:15.240739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1316/2645] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:15.245175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.306772 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78339f-2e2d-43ee-b96b-c78f8557f0b2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.311351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.311956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.312237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.312513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.313234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.315812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.393919 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1330/2646] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:52:15 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:15.398141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d63209-509c-425d-9076-72054f8d5520 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.409577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d63209-509c-425d-9076-72054f8d5520 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.464884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.465566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.465798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.466072 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.466725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.468655 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.557858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1317/2647] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:52:15 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:15.561783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.573043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee83e1-715c-45c5-857a-62078359e6e8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.577720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.578338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.578604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.578879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.580535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.582170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.592479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1331/2648] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:15.597227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.607300 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a047e-24d9-49bd-877d-03bdf3492e44 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.612651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.613339 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.613621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.613896 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] PATH_INFO: `/v3/projects/962c8649f6c9462280519b725a685b21/users/77a52987c0b7453093d510a0fe2da03c/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.628748 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=962c8649f6c9462280519b725a685b21, user_id=77a52987c0b7453093d510a0fe2da03c, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.631453 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.690316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1318/2649] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:15 2022] PUT /identity/v3/projects/962c8649f6c9462280519b725a685b21/users/77a52987c0b7453093d510a0fe2da03c/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:15.694937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.704694 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9770-35af-475c-9964-c9acf9d1e8cb admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.723824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:15.759975 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c2219-a1b6-4b21-a425-353f6b1141e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.760578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c2219-a1b6-4b21-a425-353f6b1141e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.760855 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c2219-a1b6-4b21-a425-353f6b1141e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.761134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c2219-a1b6-4b21-a425-353f6b1141e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.761845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c2219-a1b6-4b21-a425-353f6b1141e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.763419 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c2219-a1b6-4b21-a425-353f6b1141e2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.773435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1332/2650] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:15.777600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.783433 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:15.788838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c768da-4ed2-44cf-9896-bdd54c78d3d9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.793441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.794090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.794369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.794639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] PATH_INFO: `/v3/projects/962c8649f6c9462280519b725a685b21/users/77a52987c0b7453093d510a0fe2da03c/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.796618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=962c8649f6c9462280519b725a685b21, user_id=77a52987c0b7453093d510a0fe2da03c, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:15.799873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:15.809996 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1319/2651] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:15 2022] PUT /identity/v3/projects/962c8649f6c9462280519b725a685b21/users/77a52987c0b7453093d510a0fe2da03c/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:15.821856 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.822139 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.822410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:15.849906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] MFA Rules not processed for user `77a52987c0b7453093d510a0fe2da03c`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:15.851144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:15.919560 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1333/2652] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:15.931503 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:15.933278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:15.992418 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefca2d-d75a-40e4-a76f-ad4e83881b3f admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:15.996099 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:15.996721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:15.997006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:15.997311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:16.006749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:16.008199 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:16.018206 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1320/2653] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:16.253851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:16.263252 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3d872-4fb9-4e5a-9c81-920feaf2dd0b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:16.267409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf8313-9235-4fe3-8838-655665758a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:16.268008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8313-9235-4fe3-8838-655665758a48 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:16.268285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8313-9235-4fe3-8838-655665758a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:16.268547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf8313-9235-4fe3-8838-655665758a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:16.323860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf8313-9235-4fe3-8838-655665758a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:16.325316 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf8313-9235-4fe3-8838-655665758a48 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:16.335051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1334/2654] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:26.013367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:26.330852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=80 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:29.133046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:29.197999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a056f54-0426-4d85-bf40-86691a946084 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:29.202741 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:29.203401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:29.203688 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:29.203962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] PATH_INFO: `/v3/users/4ad0734fdebe48a2b21357857777728d` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:29.205125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad0734fdebe48a2b21357857777728d)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:29.207085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:29.326452 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad0734fdebe48a2b21357857777728d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:29.327102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad0734fdebe48a2b21357857777728d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:29.327716 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad0734fdebe48a2b21357857777728d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:29.340139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad0734fdebe48a2b21357857777728d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:29.381015 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad0734fdebe48a2b21357857777728d'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:29.393166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1321/2655] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:29 2022] DELETE /identity/v3/users/4ad0734fdebe48a2b21357857777728d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:29.398080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:29.398590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:29.399070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:29.418650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:29.419837 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:29.477005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1335/2656] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:29.487014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:29.488397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:29.546832 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f42719-a05f-4426-809a-081d19229706 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:29.550541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:29.551155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:29.551440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:29.551719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:29.561254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:29.562705 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:29.573524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1322/2657] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:30.031633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:30.033357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:30.048483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5ef8d-df63-4602-9ad0-b302f55e4d76 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:30.053119 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:30.053813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:30.054625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:30.054903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] PATH_INFO: `/v3/projects/bbe237eca964481396a599237889cc7a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:30.056085 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe237eca964481396a599237889cc7a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:30.060201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:30.152901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe237eca964481396a599237889cc7a', 'request_id': 'req-01fed67f-2194-4588-afff-d806fb31324a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.159045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe237eca964481396a599237889cc7a', 'request_id': 'req-01fed67f-2194-4588-afff-d806fb31324a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.198501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe237eca964481396a599237889cc7a', 'request_id': 'req-01fed67f-2194-4588-afff-d806fb31324a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.209618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] The token cache is being invalidate because project bbe237eca964481396a599237889cc7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:30.209906 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.210564 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.212109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1336/2658] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:30 2022] DELETE /identity/v3/projects/bbe237eca964481396a599237889cc7a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:30.216076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:30.225702 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4755215-f5a0-4e8d-92b7-a5bd43164f0d service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:30.245681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:30.284350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:30.285756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:30.286008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:30.286290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] PATH_INFO: `/v3/users/77a52987c0b7453093d510a0fe2da03c` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:30.287699 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a52987c0b7453093d510a0fe2da03c)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:30.290408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:30.482117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a52987c0b7453093d510a0fe2da03c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.482781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a52987c0b7453093d510a0fe2da03c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.483402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a52987c0b7453093d510a0fe2da03c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.495381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a52987c0b7453093d510a0fe2da03c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.574131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-80a58b71-e932-4edf-99d4-c6aea1b40f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a52987c0b7453093d510a0fe2da03c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:30.587205 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1323/2659] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:30 2022] DELETE /identity/v3/users/77a52987c0b7453093d510a0fe2da03c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:31.285500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:31.287886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:31.300730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fed67f-2194-4588-afff-d806fb31324a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:31.368105 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:31.368810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:31.369089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:31.369392 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] PATH_INFO: `/v3/projects/962c8649f6c9462280519b725a685b21` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:31.370548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=962c8649f6c9462280519b725a685b21)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:31.372449 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:31.457279 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:31.473605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962c8649f6c9462280519b725a685b21', 'request_id': 'req-f63b3ea6-9369-48f0-b09c-cb80269845d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:31.480675 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962c8649f6c9462280519b725a685b21', 'request_id': 'req-f63b3ea6-9369-48f0-b09c-cb80269845d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:31.541637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962c8649f6c9462280519b725a685b21', 'request_id': 'req-f63b3ea6-9369-48f0-b09c-cb80269845d8'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:31.555121 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] The token cache is being invalidate because project 962c8649f6c9462280519b725a685b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:31.555428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:31.556759 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f63b3ea6-9369-48f0-b09c-cb80269845d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:31.558481 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1337/2660] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:31 2022] DELETE /identity/v3/projects/962c8649f6c9462280519b725a685b21 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:31.563434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:31.563910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:31.564165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:31.586274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:31.587667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:31.647219 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1324/2661] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:52:31.655302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:31.655584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:31.655857 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:31.677611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:31.678821 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:31.688364 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1338/2662] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:31.692546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:31.702538 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25903f2e-a94e-461b-96d1-9d92b819acf6 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:31.707141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:31.707802 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:31.708033 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:31.708308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:31.709032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:52:31.709376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:31.711117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:31.722624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1325/2663] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:52:31.728212 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:31.737776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b4119-43f5-47ae-965a-182680c5e646 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:31.742963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:31.743556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:31.743831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:31.744100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:31.744767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:31.746723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:31.862984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1339/2664] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:52:31 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:52:31.867738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:31.877921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c9c07-9ee1-407e-84bd-8789b4774061 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:31.903561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:31.935604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:31.936217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:31.936500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:31.936770 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:31.937467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:31.939398 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.032930 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1326/2665] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:52:31 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:32.036992 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.048318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879d469-ed61-4cfa-91f1-153547b7da6d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.053247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.053888 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.054250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.054548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.055275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.056883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.067170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1340/2666] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:52:32.071081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.081092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e528f-b901-419d-8925-bd900952b242 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.086478 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9475-bbc3-4f69-93d1-0a444e86436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.087082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9475-bbc3-4f69-93d1-0a444e86436d admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.087364 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9475-bbc3-4f69-93d1-0a444e86436d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.087640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9475-bbc3-4f69-93d1-0a444e86436d admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.088331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba9475-bbc3-4f69-93d1-0a444e86436d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.089973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba9475-bbc3-4f69-93d1-0a444e86436d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.100721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1327/2667] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:32.106084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.116005 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671555ae-6f2a-421e-b1bf-d6d17de9b96a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.120908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.121600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.121880 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.122148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] PATH_INFO: `/v3/projects/9e03132e28a44d19954c6bab38e14e59/users/e59219e609894bee9ca427bda4c12bc1/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.136742 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e03132e28a44d19954c6bab38e14e59, user_id=e59219e609894bee9ca427bda4c12bc1, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.139662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.261802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1341/2668] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:32 2022] PUT /identity/v3/projects/9e03132e28a44d19954c6bab38e14e59/users/e59219e609894bee9ca427bda4c12bc1/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:32.273323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.273611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.273882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.301563 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] MFA Rules not processed for user `e59219e609894bee9ca427bda4c12bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:32.302719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:32.367576 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1328/2669] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:52:32.371968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.434148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd470612-5c78-45e8-a537-3c3323b40cee admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.438673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.439275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.439553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.439822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.440495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.442456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.547827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1342/2670] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:52:32 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:52:32.556916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.564073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777ae1a-340a-40c1-a6d6-0fd9278bc6ca None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.620231 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.620879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.621160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.621468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.622160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.624081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.700894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1329/2671] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:52:32 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:32.704902 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.716148 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7e0e5-c585-45e2-b797-cbe140576860 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.720788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.721431 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.721712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.721984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.722674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.724271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.735521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1343/2672] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:32.739634 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.749923 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78039187-8e42-404c-b5b2-96cb2fa6fc56 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.754535 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.755187 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.755469 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.755745 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] PATH_INFO: `/v3/projects/a9801628424f4f9fa161f77a4ac25123/users/afba7312ede647078888aba217d13732/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.770360 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9801628424f4f9fa161f77a4ac25123, user_id=afba7312ede647078888aba217d13732, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.773636 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.844460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1330/2673] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:32 2022] PUT /identity/v3/projects/a9801628424f4f9fa161f77a4ac25123/users/afba7312ede647078888aba217d13732/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:32.848328 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.859770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e25fa2-8cf8-44ba-af19-ec0d98d63945 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.879196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:32.917219 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc26aa4-a762-4808-8430-cb86b202cfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.917846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26aa4-a762-4808-8430-cb86b202cfdc admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.918152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26aa4-a762-4808-8430-cb86b202cfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.918413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc26aa4-a762-4808-8430-cb86b202cfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.919099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc26aa4-a762-4808-8430-cb86b202cfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.920687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc26aa4-a762-4808-8430-cb86b202cfdc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.930730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1344/2674] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:32.934995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:32.940445 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:32.945890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9711fe0-5822-426b-8bd6-b9692c9defff admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:32.950561 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:32.951217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.951499 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.951778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] PATH_INFO: `/v3/projects/a9801628424f4f9fa161f77a4ac25123/users/afba7312ede647078888aba217d13732/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:32.953781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9801628424f4f9fa161f77a4ac25123, user_id=afba7312ede647078888aba217d13732, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:32.956440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:32.967314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1331/2675] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:32 2022] PUT /identity/v3/projects/a9801628424f4f9fa161f77a4ac25123/users/afba7312ede647078888aba217d13732/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:32.978222 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:32.978542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:32.978819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:33.006349 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] MFA Rules not processed for user `afba7312ede647078888aba217d13732`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:33.007621 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:33.077075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1345/2676] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:33.088976 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:33.090715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:33.149687 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2857a-3898-4fe1-9acc-5cbbb0cf0fbd admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:33.153352 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:33.153962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:33.154244 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:33.154519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:33.212571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:33.214189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:33.223236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1332/2677] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:38.217588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:38.219088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:38.231984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e0343-60da-4ff3-842e-a2d735a3d4ff None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:38.235534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33352be5-ce3f-41f1-9d60-230fcb28b0e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:38.236137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33352be5-ce3f-41f1-9d60-230fcb28b0e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:38.236413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33352be5-ce3f-41f1-9d60-230fcb28b0e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:38.236681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33352be5-ce3f-41f1-9d60-230fcb28b0e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:38.245633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33352be5-ce3f-41f1-9d60-230fcb28b0e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:38.247051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33352be5-ce3f-41f1-9d60-230fcb28b0e7 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:38.256264 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1346/2678] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:43.219056 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:44.031631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:44.097882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee948bf5-d2d2-45f2-9c2a-90a33ce5fafb service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:44.103094 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:44.103772 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:44.104053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:44.104898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] PATH_INFO: `/v3/users/e59219e609894bee9ca427bda4c12bc1` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:44.106307 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59219e609894bee9ca427bda4c12bc1)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:44.108480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:44.298001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59219e609894bee9ca427bda4c12bc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:44.298670 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59219e609894bee9ca427bda4c12bc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:44.299161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59219e609894bee9ca427bda4c12bc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:44.312226 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59219e609894bee9ca427bda4c12bc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:44.381555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59219e609894bee9ca427bda4c12bc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:44.395651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1333/2679] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:44 2022] DELETE /identity/v3/users/e59219e609894bee9ca427bda4c12bc1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:44.400032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:44.400491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:44.400632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:44.421152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:44.422410 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:44.500373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1347/2680] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:44.510288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:44.512301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:44.598070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd448dd-9bdd-4513-8785-f8edc658f716 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:44.601762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:44.602645 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:44.602934 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:44.603603 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:44.615432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:44.617190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:44.630124 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1334/2681] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:45.292828 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:45.294484 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:45.308972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85dcd4b-a581-4b58-8fb4-ae0fd287f70f None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:45.314499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:45.315160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:45.315450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:45.315725 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] PATH_INFO: `/v3/projects/9e03132e28a44d19954c6bab38e14e59` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:45.317527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e03132e28a44d19954c6bab38e14e59)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:45.319965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:45.429385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:45.446485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e03132e28a44d19954c6bab38e14e59', 'request_id': 'req-dfb81675-96ea-4bc7-96e9-7a39aad82736'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.454849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e03132e28a44d19954c6bab38e14e59', 'request_id': 'req-dfb81675-96ea-4bc7-96e9-7a39aad82736'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.535381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e03132e28a44d19954c6bab38e14e59', 'request_id': 'req-dfb81675-96ea-4bc7-96e9-7a39aad82736'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.547615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] The token cache is being invalidate because project 9e03132e28a44d19954c6bab38e14e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:45.548276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.548941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.550915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1348/2682] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:45 2022] DELETE /identity/v3/projects/9e03132e28a44d19954c6bab38e14e59 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:45.554592 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:45.568416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346611af-8cb1-4b58-b020-4dd1897a614e service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:45.588285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:45.639124 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:45.645617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:45.649230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:45.649230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] PATH_INFO: `/v3/users/afba7312ede647078888aba217d13732` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:45.649230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afba7312ede647078888aba217d13732)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:45.649790 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:45.795068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afba7312ede647078888aba217d13732'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.795739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afba7312ede647078888aba217d13732'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.796356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afba7312ede647078888aba217d13732'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.809115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afba7312ede647078888aba217d13732'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.883933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-9fcf9719-1f5e-46cc-9826-6a5bf7938261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afba7312ede647078888aba217d13732'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:45.905297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1335/2683] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:45 2022] DELETE /identity/v3/users/afba7312ede647078888aba217d13732 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:46.394507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:46.394656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:46.410997 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb81675-96ea-4bc7-96e9-7a39aad82736 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:46.493523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:46.494202 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:46.494453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:46.494720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] PATH_INFO: `/v3/projects/a9801628424f4f9fa161f77a4ac25123` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:46.495985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9801628424f4f9fa161f77a4ac25123)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:46.497946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:46.632286 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9801628424f4f9fa161f77a4ac25123', 'request_id': 'req-8894f678-0302-4119-aee8-2a4a7cc76e79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:46.638588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9801628424f4f9fa161f77a4ac25123', 'request_id': 'req-8894f678-0302-4119-aee8-2a4a7cc76e79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:46.724153 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9801628424f4f9fa161f77a4ac25123', 'request_id': 'req-8894f678-0302-4119-aee8-2a4a7cc76e79'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:46.736399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] The token cache is being invalidate because project a9801628424f4f9fa161f77a4ac25123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:46.736704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:46.737356 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-8894f678-0302-4119-aee8-2a4a7cc76e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:46.738878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1349/2684] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:46 2022] DELETE /identity/v3/projects/a9801628424f4f9fa161f77a4ac25123 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:46.745683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:46.746163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:46.746402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:46.766541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:46.767733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:46.828690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1336/2685] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:46.837074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:46.837386 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:46.837659 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:46.857782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:46.858957 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:46.866649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1350/2686] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:52:46.871230 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:46.880848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6dc4b7-ace5-4627-9052-b41bc89ef6a0 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:46.885434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:46.886043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:46.886324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:46.886997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:46.887293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:52:46.887589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:46.889216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:46.901551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1337/2687] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:52:46.905962 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:46.915227 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0ffad-82fc-4500-a2af-ba40a3764645 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:46.920680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:46.921304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:46.921584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:46.921851 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:46.922515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:46.924413 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:47.015188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1351/2688] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:52:46 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:47.016556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:47.026739 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603971-d161-4111-8cf1-bbbe2b24ccf0 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:47.053125 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:47.084292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:47.084907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:47.085193 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:47.085504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:47.086171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:47.088262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:47.169633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1338/2689] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:52:47 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:52:47.173812 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:47.186158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa2bab-a6b4-4dfa-9a12-207988c73f02 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:47.191480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:47.192272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:47.192536 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:47.192720 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:47.193553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:47.195144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:47.206640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1352/2690] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:52:47.210823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:47.223185 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136809f6-a4e7-467f-9617-001989d1ce0d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:47.228378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68392bfc-1ce1-47f5-938d-c3f274961c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:47.229141 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68392bfc-1ce1-47f5-938d-c3f274961c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:47.230164 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68392bfc-1ce1-47f5-938d-c3f274961c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:47.230449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68392bfc-1ce1-47f5-938d-c3f274961c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:47.231280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68392bfc-1ce1-47f5-938d-c3f274961c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:47.232905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68392bfc-1ce1-47f5-938d-c3f274961c71 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:47.247229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1339/2691] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:52:47.248489 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:47.259570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95128e91-df2b-4762-86c6-32bd72cad40d admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:47.264802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:47.265500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:47.265782 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:47.266057 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] PATH_INFO: `/v3/projects/b89585eb49ee46b69f208f5500fce3ae/users/a1bc8f72c3ee4a8a8b6ebdaa03306a90/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:47.268439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:47.282718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89585eb49ee46b69f208f5500fce3ae, user_id=a1bc8f72c3ee4a8a8b6ebdaa03306a90, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:47.285411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:47.387091 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1353/2692] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:52:47 2022] PUT /identity/v3/projects/b89585eb49ee46b69f208f5500fce3ae/users/a1bc8f72c3ee4a8a8b6ebdaa03306a90/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:47.462691 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:47.462982 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:47.463267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:47.491361 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] MFA Rules not processed for user `a1bc8f72c3ee4a8a8b6ebdaa03306a90`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:47.492557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:47.558935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1340/2693] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:52:47.569845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:47.571521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:47.627840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60718303-7f79-4207-b7cc-057592e85855 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:47.631609 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:47.632218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:47.632498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:47.632769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:47.641308 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:47.642738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:47.651746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1354/2694] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:50.599597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:50.601008 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:50.660197 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525856a3-8cdc-435c-8a9c-213d9d9908c7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:50.663786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:50.664401 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:50.664685 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:50.664954 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:50.667473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:50.714723 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:50.730068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:50.731526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:50.741308 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1341/2695] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:54.362893 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:54.364294 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:54.442232 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261697c7-0f4e-4911-9ee6-bc94e417fdab service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:54.447188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:54.447834 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:54.448432 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:54.456523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:54.456523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:54.461690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:54.463255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:54.474114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1355/2696] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:52:58.374423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:58.375997 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:58.388447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5f57d-31ec-4810-90c1-90998bb2edc8 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:58.393069 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:58.393746 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:58.394028 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:58.394302 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] PATH_INFO: `/v3/users/ac48e585c133487dadbe0a1e095bfde0` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:58.408830 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac48e585c133487dadbe0a1e095bfde0)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:58.410828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:58.593697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac48e585c133487dadbe0a1e095bfde0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:58.594329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac48e585c133487dadbe0a1e095bfde0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:58.594929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac48e585c133487dadbe0a1e095bfde0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:58.607862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac48e585c133487dadbe0a1e095bfde0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:58.678311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac48e585c133487dadbe0a1e095bfde0'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:58.692011 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:58.695798 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1342/2697] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:58 2022] DELETE /identity/v3/users/ac48e585c133487dadbe0a1e095bfde0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:52:58.704994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd02803-1227-4b39-97bb-af54ecc16ca7 service placement] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:58.762556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:58.763189 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:58.763466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:58.763734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] PATH_INFO: `/v3/users/a1bc8f72c3ee4a8a8b6ebdaa03306a90` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:58.764915 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1bc8f72c3ee4a8a8b6ebdaa03306a90)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:58.766864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:58.782152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:59.001664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:59.003188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.009109 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1bc8f72c3ee4a8a8b6ebdaa03306a90'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.016946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b77bf-3abf-4d93-a3c3-133337287a44 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:59.019450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1bc8f72c3ee4a8a8b6ebdaa03306a90'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.020409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1bc8f72c3ee4a8a8b6ebdaa03306a90'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.021600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:59.022242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:59.024862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:59.024862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] PATH_INFO: `/v3/projects/32813adc26c64016b99c9c5fd838d391` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:59.036932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1bc8f72c3ee4a8a8b6ebdaa03306a90'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.038501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32813adc26c64016b99c9c5fd838d391)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:59.040518 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:59.094996 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4656eae3-7db6-4a7b-b043-baef1cd4e3d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1bc8f72c3ee4a8a8b6ebdaa03306a90'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.101534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1356/2698] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:58 2022] DELETE /identity/v3/users/a1bc8f72c3ee4a8a8b6ebdaa03306a90 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:52:59.106687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:59.106978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:59.107257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:59.126669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:52:59.128022 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.142257 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32813adc26c64016b99c9c5fd838d391', 'request_id': 'req-0fb6a046-5572-4877-8df5-ba447ebf2db1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.148426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32813adc26c64016b99c9c5fd838d391', 'request_id': 'req-0fb6a046-5572-4877-8df5-ba447ebf2db1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.189478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1357/2699] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:52:59.199297 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:59.200603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.215410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32813adc26c64016b99c9c5fd838d391', 'request_id': 'req-0fb6a046-5572-4877-8df5-ba447ebf2db1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.227256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] The token cache is being invalidate because project 32813adc26c64016b99c9c5fd838d391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:52:59.227544 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.228180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.230448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1343/2700] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:59 2022] DELETE /identity/v3/projects/32813adc26c64016b99c9c5fd838d391 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:52:59.234295 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:59.235650 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.248572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6a046-5572-4877-8df5-ba447ebf2db1 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:59.253470 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:59.254694 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:59.254988 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:59.255267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] PATH_INFO: `/v3/users/73471e443c784730821030edabb8804f` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:59.259089 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3dd7f-331b-49cf-87d0-99eeb75b851a None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:59.263808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:59.264424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:59.264705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:59.265406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:59.266789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.270820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73471e443c784730821030edabb8804f)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:59.272787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:59.278575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:59.280017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:59.281461 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:59.295840 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1358/2701] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:52:59.457657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73471e443c784730821030edabb8804f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.458251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73471e443c784730821030edabb8804f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.458893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73471e443c784730821030edabb8804f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.471067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73471e443c784730821030edabb8804f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.546862 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73471e443c784730821030edabb8804f'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.561795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1344/2702] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:52:59 2022] DELETE /identity/v3/users/73471e443c784730821030edabb8804f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:52:59.765695 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:59.767081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.781504 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494ffcac-1223-496c-8081-63a2bb522883 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:59.794834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:52:59.796273 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:52:59.830500 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:52:59.855197 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:59.855924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:59.856271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:59.856639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] PATH_INFO: `/v3/projects/b89585eb49ee46b69f208f5500fce3ae` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:59.857916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89585eb49ee46b69f208f5500fce3ae)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:59.859898 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:59.872590 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7699a-221c-42b1-9410-d26237b02659 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:52:59.877321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:52:59.877957 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:52:59.878236 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:52:59.878511 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] PATH_INFO: `/v3/projects/19fd5873e36042d79e2ee534175828f5` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:52:59.890086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fd5873e36042d79e2ee534175828f5)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:52:59.892029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:52:59.962435 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89585eb49ee46b69f208f5500fce3ae', 'request_id': 'req-b2452ba2-a027-4280-942e-62a0d6265893'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.969158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89585eb49ee46b69f208f5500fce3ae', 'request_id': 'req-b2452ba2-a027-4280-942e-62a0d6265893'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:52:59.999254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fd5873e36042d79e2ee534175828f5', 'request_id': 'req-44041f14-f630-4ad1-bcd6-b72b96f5e415'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.005901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fd5873e36042d79e2ee534175828f5', 'request_id': 'req-44041f14-f630-4ad1-bcd6-b72b96f5e415'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.060670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89585eb49ee46b69f208f5500fce3ae', 'request_id': 'req-b2452ba2-a027-4280-942e-62a0d6265893'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.077429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] The token cache is being invalidate because project b89585eb49ee46b69f208f5500fce3ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:53:00.077827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.078639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b2452ba2-a027-4280-942e-62a0d6265893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.081074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1359/2703] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:59 2022] DELETE /identity/v3/projects/b89585eb49ee46b69f208f5500fce3ae => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:00.086728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.087020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.088993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.105283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fd5873e36042d79e2ee534175828f5', 'request_id': 'req-44041f14-f630-4ad1-bcd6-b72b96f5e415'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.108030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:00.109244 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:00.116708 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] The token cache is being invalidate because project 19fd5873e36042d79e2ee534175828f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:53:00.117000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.117652 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-44041f14-f630-4ad1-bcd6-b72b96f5e415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:00.119163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1345/2704] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:52:59 2022] DELETE /identity/v3/projects/19fd5873e36042d79e2ee534175828f5 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:00.175487 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1360/2705] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:53:00.184579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.184995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.185395 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.195787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:00.209442 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:00.210684 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:00.223998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1346/2706] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:00.224974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.235770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c1521-2923-414b-83c4-95ef051b97de None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.241488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.242235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.242515 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.242786 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.243561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:53:00.243862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.245746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:00.258921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1361/2707] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:53:00.264459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.274568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c3c65-9b1f-4dac-9568-09f29c3663ab None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.281329 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.282119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.282407 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.282686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.283369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.289385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:00.474131 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1347/2708] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:53:00 2022] POST /identity/v3/projects => generated 375 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:53:00.478646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.489665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fcbd8-26e0-4ee6-a00b-838427d1151c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.539625 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:00.547221 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.552687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.552687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.552687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.552687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.552687 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:00.619131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1362/2709] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:53:00 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:53:00.623540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.633794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddebad4-7382-48ea-81a2-8effb1151ead admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.638504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.639105 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.639386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.639658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.640348 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.641995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:00.652616 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1348/2710] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:00.656499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.666764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a022b32-2713-4dc1-808a-6989cbf0a34f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.672041 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb4da-3bb4-48a7-b59d-5d498b21a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.674185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb4da-3bb4-48a7-b59d-5d498b21a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.674185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb4da-3bb4-48a7-b59d-5d498b21a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.674185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb4da-3bb4-48a7-b59d-5d498b21a4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.674185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb4da-3bb4-48a7-b59d-5d498b21a4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.675648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb4da-3bb4-48a7-b59d-5d498b21a4e0 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:00.686288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1363/2711] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:53:00.690582 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.700911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67743ecd-bbfb-44aa-8a3d-a71a9526db7d admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.705472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.706119 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.706397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.706671 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] PATH_INFO: `/v3/projects/6e98c739b68d486898155e7f757785c5/users/8c0fb6a52fbc4e6688f095be1bf9d2ac/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.721421 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e98c739b68d486898155e7f757785c5, user_id=8c0fb6a52fbc4e6688f095be1bf9d2ac, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.724086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:00.785065 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1349/2712] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:53:00 2022] PUT /identity/v3/projects/6e98c739b68d486898155e7f757785c5/users/8c0fb6a52fbc4e6688f095be1bf9d2ac/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:00.796799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.797379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.797492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.825285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] MFA Rules not processed for user `8c0fb6a52fbc4e6688f095be1bf9d2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:00.826492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:00.889796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1364/2713] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:00.894386 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:00.955388 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfa5c4-9d4c-49ed-8086-b444ccdf2285 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:00.959999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:00.960608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:00.960895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:00.961170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:00.961868 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:00.963781 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.079638 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1350/2714] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:53:00 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:53:01.082729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.094338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d776371-a147-4d7d-9071-7fada8cb0ab8 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.150287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.150894 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.151172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.151443 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.152384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.154734 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.226731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1365/2715] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:53:01 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:53:01.231188 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.241491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99434008-ac63-48e6-952f-e125d70b8d2b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.246689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.247290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.247568 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.247839 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.248528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.250162 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.260555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1351/2716] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:01.264800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.274568 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592874dd-e586-4067-9b4f-151ab1a170d2 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.280815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cc432-9cd3-4075-99cd-42693937edf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.281446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc432-9cd3-4075-99cd-42693937edf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.281722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc432-9cd3-4075-99cd-42693937edf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.281995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cc432-9cd3-4075-99cd-42693937edf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.282678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc432-9cd3-4075-99cd-42693937edf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.284278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cc432-9cd3-4075-99cd-42693937edf2 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.294832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1366/2717] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:01.298950 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.308899 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31339-61ba-4897-af38-6ae90a01c80a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.313577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.314229 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.314514 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.314799 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] PATH_INFO: `/v3/projects/c64353f2ba8a43be8fda7e23970194ea/users/20d3634c751049638209d382cc4f7396/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.329092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64353f2ba8a43be8fda7e23970194ea, user_id=20d3634c751049638209d382cc4f7396, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.331838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.404089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1352/2718] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:53:01 2022] PUT /identity/v3/projects/c64353f2ba8a43be8fda7e23970194ea/users/20d3634c751049638209d382cc4f7396/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:01.414275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.414561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.414835 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.442523 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] MFA Rules not processed for user `20d3634c751049638209d382cc4f7396`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:01.443723 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:01.474406 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:01.508194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1367/2719] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:01.513311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.564010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:01.575714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a241dc7-6929-41c0-bdf4-44bd69acb6f8 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.580432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.581041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.581356 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.581658 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.582369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.584282 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.677689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1353/2720] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:53:01 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:53:01.681745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.692729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980f20c-8dfb-4dc1-be89-911d2b4826cb None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.746974 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.747567 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.747844 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.748114 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.748778 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.750719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.859862 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1368/2721] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:53:01 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:53:01.863960 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.875117 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b51b7-3ca0-4aec-9e53-2515b286a495 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.879794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.880394 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.880678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.880955 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.881697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.886895 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.893986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1354/2722] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:01.898010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:01.908240 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2e991-721e-4020-83c7-0b83b51dfb6e admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:01.912860 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:01.913590 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:01.913873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:01.914147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] PATH_INFO: `/v3/projects/2260c674e2b04d98b99a91c709f0b9ef/users/480fdbbb545c40fcb1350e0912928572/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:01.930032 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2260c674e2b04d98b99a91c709f0b9ef, user_id=480fdbbb545c40fcb1350e0912928572, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:01.932771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:01.998571 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1369/2723] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:53:01 2022] PUT /identity/v3/projects/2260c674e2b04d98b99a91c709f0b9ef/users/480fdbbb545c40fcb1350e0912928572/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:02.002653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:02.013368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2e583-9d79-4d6e-9dba-63cafa6ccdfe admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:02.067534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bb229-3599-4350-b4fd-8483c18c1658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:02.068151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bb229-3599-4350-b4fd-8483c18c1658 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:02.068439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bb229-3599-4350-b4fd-8483c18c1658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:02.068715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bb229-3599-4350-b4fd-8483c18c1658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:02.069439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bb229-3599-4350-b4fd-8483c18c1658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:02.071195 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bb229-3599-4350-b4fd-8483c18c1658 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:02.082911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1355/2724] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:02.087444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:02.098103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d5e6d-c3b2-4a9c-8fd2-a16648f815e5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:02.103348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:02.104662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:02.104662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:02.104662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] PATH_INFO: `/v3/projects/2260c674e2b04d98b99a91c709f0b9ef/users/480fdbbb545c40fcb1350e0912928572/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:02.107561 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2260c674e2b04d98b99a91c709f0b9ef, user_id=480fdbbb545c40fcb1350e0912928572, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:02.110340 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:02.121437 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1370/2725] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:53:02 2022] PUT /identity/v3/projects/2260c674e2b04d98b99a91c709f0b9ef/users/480fdbbb545c40fcb1350e0912928572/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:02.131609 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:02.131877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:02.132155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:02.161092 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] MFA Rules not processed for user `480fdbbb545c40fcb1350e0912928572`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:02.162345 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:02.232029 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1356/2726] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:02.243271 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:02.244970 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:02.301972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8618858-c514-4232-aa9a-ed7a2b5d605c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:02.305624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:02.306228 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:02.306505 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:02.306776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:02.345581 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:02.364267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:02.365729 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:02.375132 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1371/2727] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:04.282656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:04.284087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:04.296754 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09c761-2f0b-44e9-b287-f29ed795ac14 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:04.300321 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:04.300928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:04.301238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:04.301528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:04.345190 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:04.358263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:04.359718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:04.368423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1357/2728] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:05.734950 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:05.743460 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741247aa-672a-4cb0-bf92-b787d3e5675e service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:05.747065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed2e64-77ab-462e-b06e-2aecda1423ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:05.747664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed2e64-77ab-462e-b06e-2aecda1423ea service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:05.747941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed2e64-77ab-462e-b06e-2aecda1423ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:05.748220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed2e64-77ab-462e-b06e-2aecda1423ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:05.757479 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed2e64-77ab-462e-b06e-2aecda1423ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:05.758918 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed2e64-77ab-462e-b06e-2aecda1423ea service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:05.768727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1372/2729] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:10.275414 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:10.337808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c208c-e989-492a-b8b3-4e134adafe01 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:10.342456 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:10.343089 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:10.343376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:10.344167 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] PATH_INFO: `/v3/users/8c0fb6a52fbc4e6688f095be1bf9d2ac` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:10.345586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0fb6a52fbc4e6688f095be1bf9d2ac)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:10.347541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:10.484964 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0fb6a52fbc4e6688f095be1bf9d2ac'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:10.485586 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0fb6a52fbc4e6688f095be1bf9d2ac'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:10.486196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0fb6a52fbc4e6688f095be1bf9d2ac'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:10.497887 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0fb6a52fbc4e6688f095be1bf9d2ac'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:10.551555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0fb6a52fbc4e6688f095be1bf9d2ac'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:10.564653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1358/2730] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:53:10 2022] DELETE /identity/v3/users/8c0fb6a52fbc4e6688f095be1bf9d2ac => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:10.569160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:10.575771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:10.575771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:10.588827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:10.590054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:10.648054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1373/2731] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:53:10.658301 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:10.659731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:10.842408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71021aa7-be5b-4e9f-8227-1dbe63b243fb admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:10.847073 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:10.847704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:10.847965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:10.848233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:10.857463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:10.858898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:10.869102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1359/2732] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:11.244775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:11.246218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:11.260510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100e9f4-40a8-4762-b772-efa42028b2b8 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:11.265391 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:11.266043 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:11.266516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:11.266796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] PATH_INFO: `/v3/projects/6e98c739b68d486898155e7f757785c5` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:11.268064 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e98c739b68d486898155e7f757785c5)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:11.270055 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:11.370000 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e98c739b68d486898155e7f757785c5', 'request_id': 'req-137f00b5-e882-45a8-8631-60973bf3dfd7'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.376865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e98c739b68d486898155e7f757785c5', 'request_id': 'req-137f00b5-e882-45a8-8631-60973bf3dfd7'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.442020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e98c739b68d486898155e7f757785c5', 'request_id': 'req-137f00b5-e882-45a8-8631-60973bf3dfd7'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.452799 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] The token cache is being invalidate because project 6e98c739b68d486898155e7f757785c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:53:11.453096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.453741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.455254 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1374/2733] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:53:11 2022] DELETE /identity/v3/projects/6e98c739b68d486898155e7f757785c5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:11.459492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:11.469865 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2583e06-a36c-4d93-80f2-2a4ec69d1b98 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:11.473543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:11.525681 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:11.526240 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:11.526522 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:11.527468 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] PATH_INFO: `/v3/users/20d3634c751049638209d382cc4f7396` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:11.528929 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d3634c751049638209d382cc4f7396)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:11.531935 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:11.667931 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d3634c751049638209d382cc4f7396'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.668575 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d3634c751049638209d382cc4f7396'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.669170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d3634c751049638209d382cc4f7396'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.681818 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d3634c751049638209d382cc4f7396'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.722847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d3634c751049638209d382cc4f7396'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:11.736057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1360/2734] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:53:11 2022] DELETE /identity/v3/users/20d3634c751049638209d382cc4f7396 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:12.142630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:12.144120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:12.159020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f00b5-e882-45a8-8631-60973bf3dfd7 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:12.205884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:12.217535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:12.218185 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:12.218465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:12.219427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] PATH_INFO: `/v3/projects/c64353f2ba8a43be8fda7e23970194ea` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:12.220565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64353f2ba8a43be8fda7e23970194ea)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:12.222533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:12.324048 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64353f2ba8a43be8fda7e23970194ea', 'request_id': 'req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.330740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64353f2ba8a43be8fda7e23970194ea', 'request_id': 'req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.368969 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64353f2ba8a43be8fda7e23970194ea', 'request_id': 'req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.379824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] The token cache is being invalidate because project c64353f2ba8a43be8fda7e23970194ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:53:12.380107 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.380730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.382235 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1375/2735] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:53:12 2022] DELETE /identity/v3/projects/c64353f2ba8a43be8fda7e23970194ea => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:12.386285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:12.387605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:12.400877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045bc61-a1d4-4c67-b42a-b605f5690949 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:12.456271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:12.456907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:12.457191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:12.457492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] PATH_INFO: `/v3/users/480fdbbb545c40fcb1350e0912928572` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:12.458637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] RBAC: Authorizing `identity:delete_user(user_id=480fdbbb545c40fcb1350e0912928572)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:12.460548 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:12.542528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:12.633880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '480fdbbb545c40fcb1350e0912928572'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.634534 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '480fdbbb545c40fcb1350e0912928572'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.635155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '480fdbbb545c40fcb1350e0912928572'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.647565 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '480fdbbb545c40fcb1350e0912928572'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.746086 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-74ba3881-e22a-4ee0-b4ad-842f9dba311e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '480fdbbb545c40fcb1350e0912928572'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:12.758788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1361/2736] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:53:12 2022] DELETE /identity/v3/users/480fdbbb545c40fcb1350e0912928572 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:13.164718 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.168404 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:13.181480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea75c1-e84c-4f53-ba5e-8d2f5e7d6cf5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.238701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.239350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.239633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.239908 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] PATH_INFO: `/v3/projects/2260c674e2b04d98b99a91c709f0b9ef` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.241077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2260c674e2b04d98b99a91c709f0b9ef)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.243049 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:13.364051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2260c674e2b04d98b99a91c709f0b9ef', 'request_id': 'req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:13.370088 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2260c674e2b04d98b99a91c709f0b9ef', 'request_id': 'req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:13.437656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2260c674e2b04d98b99a91c709f0b9ef', 'request_id': 'req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:13.448765 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] The token cache is being invalidate because project 2260c674e2b04d98b99a91c709f0b9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:53:13.449051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:13.449719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-7790a68f-0005-4d49-a3cd-9d0542d0c1a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:53:13.451218 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1376/2737] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:53:13 2022] DELETE /identity/v3/projects/2260c674e2b04d98b99a91c709f0b9ef => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:53:13.456458 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.456756 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.457041 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.477426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:13.478659 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:13.536517 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1362/2738] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:13.545703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.545993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.546288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.567258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:13.568492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:13.576831 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1377/2739] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:13.581324 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.590915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b40df1-9ba4-4025-bf81-2d2ff3d7c9e3 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.595559 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.596166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.596455 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.596732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.597466 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:53:13.597773 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.599374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:13.610992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1363/2740] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:53:13.615719 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.620138 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:13.625318 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879db6da-5764-4b71-a327-b4b8d3ba61d6 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.630642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.631233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.631982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.631982 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.632428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.634368 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:13.689966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1378/2741] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:53:13 2022] POST /identity/v3/projects => generated 367 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:53:13.694093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.704126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea953d-9216-427d-9e44-bbb800c562fe admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.761426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.763894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.763894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.763894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.763894 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.765541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:13.860451 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1364/2742] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:53:13 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:53:13.865137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.875459 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11050c5d-bd98-412d-9b7b-2d074cdf6383 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.880233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.880850 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.881137 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.881439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.882220 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.883810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:13.893783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1379/2743] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:13.897577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.907335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ceb7b-463c-43ca-afc9-8ce8f6e4e985 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.912667 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9794494-6d66-4b59-9166-a1af7e2c1f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.913311 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9794494-6d66-4b59-9166-a1af7e2c1f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.913576 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9794494-6d66-4b59-9166-a1af7e2c1f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.913846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9794494-6d66-4b59-9166-a1af7e2c1f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.914523 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9794494-6d66-4b59-9166-a1af7e2c1f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.916108 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9794494-6d66-4b59-9166-a1af7e2c1f42 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:13.926600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1365/2744] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:53:13.930642 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:13.940446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7430406-85b3-4f09-9772-25974f85b46a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:13.944979 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:13.945664 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:13.945942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:13.946216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] PATH_INFO: `/v3/projects/40787baf8011462ab988e12edf7c6f25/users/d8113e93c1774c45835b360cb1fafbc1/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:13.960956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40787baf8011462ab988e12edf7c6f25, user_id=d8113e93c1774c45835b360cb1fafbc1, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:13.963690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:14.032021 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1380/2745] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:53:13 2022] PUT /identity/v3/projects/40787baf8011462ab988e12edf7c6f25/users/d8113e93c1774c45835b360cb1fafbc1/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:53:14.043571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:14.043858 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:14.044132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:14.065319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:14.072305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] MFA Rules not processed for user `d8113e93c1774c45835b360cb1fafbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:14.073526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:14.138762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1366/2746] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:14.152125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:14.153826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:14.232541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922113b4-aac2-417d-8205-a0ca9f7e1e63 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:14.236705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95d7cc-db4d-438d-a325-404d5a53f50b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e4a8d38d08c4619812d98a544510b29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:14.237421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95d7cc-db4d-438d-a325-404d5a53f50b service glance] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:14.237703 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95d7cc-db4d-438d-a325-404d5a53f50b service glance] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:14.237972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95d7cc-db4d-438d-a325-404d5a53f50b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:14.246103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95d7cc-db4d-438d-a325-404d5a53f50b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:14.247525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95d7cc-db4d-438d-a325-404d5a53f50b service glance] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:14.257002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1381/2747] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:21.443933 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:21.445425 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:21.522678 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dfe06b-754c-496c-9634-6def4b645f76 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:21.528409 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:21.529032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:21.529350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:21.529627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:21.531382 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:21.543344 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:21.544952 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:21.557276 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1367/2748] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:24.252585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=58,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:25.896623 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:25.897275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:25.897395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:25.917165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] MFA Rules not processed for user `ac99f60384fe4a119d019cedfe1d3557`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:53:25.918497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:25.972613 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1382/2749] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:53:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:53:26.287742 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:26.289129 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:26.344812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee051d-3c61-4520-9d8c-ee73e8a4a0d8 service placement] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:26.348410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:26.349016 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:26.349323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:26.349598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:26.351350 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:26.361432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:26.362972 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:26.371052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1368/2750] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:35.965937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=25 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:36.367287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=85 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:45.613182 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:53:45.614710 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:45.696740 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a910e-15f8-4b1c-8597-f93da8f0b999 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:53:45.702456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:53:45.703080 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:53:45.703361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:53:45.703633 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:53:45.705629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:53:45.718087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:53:45.719678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf64651-c74d-4043-becd-b8a58e1fd094 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:53:45.730841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1383/2751] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:53:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:53:55.726326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=34 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:53:59.945809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:00.008696 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1a92f-1cc2-42d4-a235-d1a013031d32 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:00.013246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:00.013882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:00.014160 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:00.014435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] PATH_INFO: `/v3/users/d8113e93c1774c45835b360cb1fafbc1` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:00.015593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8113e93c1774c45835b360cb1fafbc1)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:00.017557 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:00.212378 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8113e93c1774c45835b360cb1fafbc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:00.212717 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8113e93c1774c45835b360cb1fafbc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:00.213268 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8113e93c1774c45835b360cb1fafbc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:00.225449 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8113e93c1774c45835b360cb1fafbc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:00.288113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8113e93c1774c45835b360cb1fafbc1'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:00.301035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1369/2752] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:53:59 2022] DELETE /identity/v3/users/d8113e93c1774c45835b360cb1fafbc1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:54:00.305513 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a028581-6554-4c12-94dc-950c64afceed None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:00.306002 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a028581-6554-4c12-94dc-950c64afceed None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:00.306111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a028581-6554-4c12-94dc-950c64afceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:00.324531 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8a028581-6554-4c12-94dc-950c64afceed None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:54:00.325746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8a028581-6554-4c12-94dc-950c64afceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:00.384589 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1384/2753] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:54:00.394284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:00.395692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:00.453112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3111a-8734-441f-9754-023d1a369e6c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:00.457106 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b063b3-75ed-4584-955b-8fceae35da86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:00.457755 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b063b3-75ed-4584-955b-8fceae35da86 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:00.458040 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b063b3-75ed-4584-955b-8fceae35da86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:00.458314 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b063b3-75ed-4584-955b-8fceae35da86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:00.467373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b063b3-75ed-4584-955b-8fceae35da86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:00.468750 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b063b3-75ed-4584-955b-8fceae35da86 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:00.478834 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1370/2754] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:54:00.888115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a028581-6554-4c12-94dc-950c64afceed None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:00.889611 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8a028581-6554-4c12-94dc-950c64afceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:00.902262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a028581-6554-4c12-94dc-950c64afceed None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:00.906896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:00.907665 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:00.907946 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:00.908226 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] PATH_INFO: `/v3/projects/40787baf8011462ab988e12edf7c6f25` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:00.909429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40787baf8011462ab988e12edf7c6f25)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:00.911366 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.014204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40787baf8011462ab988e12edf7c6f25', 'request_id': 'req-720793e7-e9fc-4052-aabe-1b4ca71011a9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:01.021042 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40787baf8011462ab988e12edf7c6f25', 'request_id': 'req-720793e7-e9fc-4052-aabe-1b4ca71011a9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:01.105276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40787baf8011462ab988e12edf7c6f25', 'request_id': 'req-720793e7-e9fc-4052-aabe-1b4ca71011a9'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:01.116343 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] The token cache is being invalidate because project 40787baf8011462ab988e12edf7c6f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:54:01.116629 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:01.117276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-720793e7-e9fc-4052-aabe-1b4ca71011a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:01.118793 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1385/2755] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:54:00 2022] DELETE /identity/v3/projects/40787baf8011462ab988e12edf7c6f25 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:54:01.124021 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.124477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.124608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.145516 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:54:01.146698 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:01.163632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:54:01.209419 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1371/2756] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:54:01.218601 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.218886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.219163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.239890 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:54:01.241051 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:01.249008 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1386/2757] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:54:01.253838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.263385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed1a9-f926-427d-b991-48ed4f96c6bf None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.268577 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.269213 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.269515 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.269791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.270482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:54:01.270786 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.272390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.284254 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1372/2758] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:54:01.287795 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.296843 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b2b15-6374-44c2-bb7d-600ef404f0c1 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.302520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.303112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.303388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.303657 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.304319 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.306288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.427792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1387/2759] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:54:01 2022] POST /identity/v3/projects => generated 399 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:54:01.431958 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.442806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c139-8c11-457d-8d50-bf35c2912aa2 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.498914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.499484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.499767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.500039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.500733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.503323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.573608 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1373/2760] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:54:01 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:54:01.577766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.588199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bfb7d-a83c-48f0-b62b-10bc9d5c4f5b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.592783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.593663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.594012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.594324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.595075 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.597447 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.607498 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1388/2761] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:54:01.611349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.622258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dc384-38d3-4ec7-bcd7-502f69cf68ac admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.627076 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b87ca4-adc1-4ab3-b8e4-5a2c0bfea01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.628334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87ca4-adc1-4ab3-b8e4-5a2c0bfea01b admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.628605 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87ca4-adc1-4ab3-b8e4-5a2c0bfea01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.628871 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b87ca4-adc1-4ab3-b8e4-5a2c0bfea01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.629602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87ca4-adc1-4ab3-b8e4-5a2c0bfea01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.631227 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b87ca4-adc1-4ab3-b8e4-5a2c0bfea01b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.641795 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1374/2762] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:54:01.645999 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.655827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273445b9-e0b4-4c16-a566-7417638feb76 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.660984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.661660 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.661936 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.662210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] PATH_INFO: `/v3/projects/2df6c0e82e0d467bbc1c907cf069143e/users/629095aed72841089a935d7be18320be/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.677084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df6c0e82e0d467bbc1c907cf069143e, user_id=629095aed72841089a935d7be18320be, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.679771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.763440 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1389/2763] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:54:01 2022] PUT /identity/v3/projects/2df6c0e82e0d467bbc1c907cf069143e/users/629095aed72841089a935d7be18320be/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:54:01.774044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.774331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.774602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.802943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] MFA Rules not processed for user `629095aed72841089a935d7be18320be`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:54:01.804130 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:01.866192 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1375/2764] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:54:01.877748 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:01.879402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:01.900886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:54:01.936501 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2f2c7-9527-4a46-b1d7-5d5d06aca154 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:01.941547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0103835-1a48-4b6d-b9bc-b3c0422f6261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:01.942155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0103835-1a48-4b6d-b9bc-b3c0422f6261 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:01.942434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0103835-1a48-4b6d-b9bc-b3c0422f6261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:01.942705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0103835-1a48-4b6d-b9bc-b3c0422f6261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:01.950968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0103835-1a48-4b6d-b9bc-b3c0422f6261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:01.952393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0103835-1a48-4b6d-b9bc-b3c0422f6261 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:01.961771 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1390/2765] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:54:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:54:10.747918 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:10.766971 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:54:10.810014 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9c24c-f7b6-428c-a368-5bb49eedb40d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:10.814713 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:10.815327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:10.815614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:10.815889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] PATH_INFO: `/v3/users/629095aed72841089a935d7be18320be` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:10.817087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629095aed72841089a935d7be18320be)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:10.819046 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:10.970247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629095aed72841089a935d7be18320be'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:10.970916 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629095aed72841089a935d7be18320be'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:10.971533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629095aed72841089a935d7be18320be'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:10.984299 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629095aed72841089a935d7be18320be'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.036271 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629095aed72841089a935d7be18320be'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.048966 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1376/2766] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:54:10 2022] DELETE /identity/v3/users/629095aed72841089a935d7be18320be => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:54:11.054580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:11.055100 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:11.055217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:11.074369 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:54:11.075658 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:11.134951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1391/2767] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:54:11.150331 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:11.151721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:11.211881 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcab6809-b1b7-4f43-8a5d-af2924acfb66 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:11.215526 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:11.216140 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:11.216421 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:11.216731 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:11.226373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:11.228182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:11.238459 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1377/2768] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:54:11.631259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:11.632768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:11.645103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c0739-37f4-43b7-8361-163f4b7c83ba None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:11.651362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:11.656068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:11.656068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:11.656068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] PATH_INFO: `/v3/projects/2df6c0e82e0d467bbc1c907cf069143e` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:11.656068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df6c0e82e0d467bbc1c907cf069143e)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:11.656068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:11.791582 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df6c0e82e0d467bbc1c907cf069143e', 'request_id': 'req-4fb03d73-e460-42fc-9558-8230fbd26d82'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.797604 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df6c0e82e0d467bbc1c907cf069143e', 'request_id': 'req-4fb03d73-e460-42fc-9558-8230fbd26d82'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.887555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df6c0e82e0d467bbc1c907cf069143e', 'request_id': 'req-4fb03d73-e460-42fc-9558-8230fbd26d82'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.899358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] The token cache is being invalidate because project 2df6c0e82e0d467bbc1c907cf069143e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:54:11.899641 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.900281 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:54:11.901810 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1392/2769] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:54:11 2022] DELETE /identity/v3/projects/2df6c0e82e0d467bbc1c907cf069143e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:54:21.233142 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:54:21.896125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:54:35.027849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:54:35.029303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:35.041267 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35554625-f832-4194-b165-ecabd1dc9b30 service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:54:35.089618 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:54:35.090246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:54:35.090553 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:54:35.090825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:54:35.092570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:54:35.103004 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:54:35.104450 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:54:35.112690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1378/2770] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:54:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:54:45.108986 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=26 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:02.311393 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:02.312776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:02.370876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb03d73-e460-42fc-9558-8230fbd26d82 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:02.374381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:02.374985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:02.375263 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:02.375534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:02.377277 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:02.449478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:02.450965 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:02.462130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1393/2771] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:10.739553 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:10.749820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8884c-1023-440d-8c4c-1829e195bf1d service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:10.753373 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:10.753978 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:10.754261 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:10.754556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:10.756509 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:10.769697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:10.771166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:10.782006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1379/2772] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:12.456903 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=54 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:20.778157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=6 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:22.169717 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:22.171267 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:22.183605 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da478da-02c9-4c50-b1a7-defd965ecca2 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:22.187194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:22.187803 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:22.188086 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:22.188361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:22.190094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:22.253656 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:22.255104 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:22.265785 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1394/2773] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:25.922498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:25.925019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:25.942381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0000ae3-3264-479a-8cbc-1c820385a547 service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:25.947505 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:25.948170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:25.948482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:25.948783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] PATH_INFO: `/v3/users/ac99f60384fe4a119d019cedfe1d3557` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:25.949963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac99f60384fe4a119d019cedfe1d3557)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:25.951876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:26.150412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac99f60384fe4a119d019cedfe1d3557'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.151068 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac99f60384fe4a119d019cedfe1d3557'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.151765 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac99f60384fe4a119d019cedfe1d3557'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.164107 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac99f60384fe4a119d019cedfe1d3557'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.227821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-55f71812-15bf-41e5-a803-037e61dc6591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac99f60384fe4a119d019cedfe1d3557'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.240998 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1380/2774] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:25 2022] DELETE /identity/v3/users/ac99f60384fe4a119d019cedfe1d3557 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:26.521480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:26.533166 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ff393-3fa4-4500-a2db-1795835e51ba service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:26.593111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:26.593781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:26.594070 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:26.594345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] PATH_INFO: `/v3/projects/a98f18ee8c7d4cfe9f4b879505ca6605` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:26.595493 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98f18ee8c7d4cfe9f4b879505ca6605)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:26.597445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:26.709951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98f18ee8c7d4cfe9f4b879505ca6605', 'request_id': 'req-b13c0743-0635-423d-8986-ef9a5cb91506'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.716007 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98f18ee8c7d4cfe9f4b879505ca6605', 'request_id': 'req-b13c0743-0635-423d-8986-ef9a5cb91506'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.783433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98f18ee8c7d4cfe9f4b879505ca6605', 'request_id': 'req-b13c0743-0635-423d-8986-ef9a5cb91506'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.794727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] The token cache is being invalidate because project a98f18ee8c7d4cfe9f4b879505ca6605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:26.795015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.795638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-b13c0743-0635-423d-8986-ef9a5cb91506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:26.797130 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1395/2775] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:26 2022] DELETE /identity/v3/projects/a98f18ee8c7d4cfe9f4b879505ca6605 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:26.802290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:26.802690 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:26.802962 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:26.822280 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:26.823463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:26.881973 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1381/2776] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:26.889917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:26.890237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:26.890510 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:26.910383 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:26.911615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:26.920270 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1396/2777] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:26.924489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:26.934501 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59cca2d-84e8-42be-b015-29aab7b2b26e None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:26.939156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:26.939757 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:26.940042 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:26.940319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:26.941013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:55:26.941370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:26.943082 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:26.954551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1382/2778] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:55:26.959269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:26.968879 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc8a1-22df-49d6-962f-3a8a40562a91 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:26.973902 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:26.975390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:26.975650 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:26.975920 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:26.976598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:26.978669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:26.996511 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:27.093421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1397/2779] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:26 2022] POST /identity/v3/projects => generated 403 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:27.097589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:27.107833 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fad4c-5af5-44e9-807b-0a1da268d8f3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:27.162910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:27.163524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:27.163801 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:27.164071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:27.164724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:27.166686 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:27.174619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:27.250637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1383/2780] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:27 2022] POST /identity/v3/users => generated 302 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:27.254767 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:27.266074 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b529f02-57c3-4586-b387-ac09370f59c8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:27.270680 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:27.272082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:27.272355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:27.272623 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:27.273342 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:27.274932 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:27.284770 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1398/2781] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:27.288724 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:27.299013 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15857a-3867-4c35-ab48-ffd211f28f03 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:27.303788 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454816c6-3430-4707-b852-c254f6e496f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:27.306408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454816c6-3430-4707-b852-c254f6e496f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:27.306408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454816c6-3430-4707-b852-c254f6e496f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:27.306408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454816c6-3430-4707-b852-c254f6e496f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:27.306408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454816c6-3430-4707-b852-c254f6e496f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:27.307235 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454816c6-3430-4707-b852-c254f6e496f5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:27.317734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1384/2782] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:27.322429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:27.332357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f42f1d-c353-425f-8ca7-c91823c9f9be admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:27.337598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:27.338245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:27.338525 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:27.338798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] PATH_INFO: `/v3/projects/c5218a711059454987d15a74f1effcdf/users/f49107b1627e48dd929016de7f93576b/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:27.353373 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5218a711059454987d15a74f1effcdf, user_id=f49107b1627e48dd929016de7f93576b, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:27.356038 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:27.435439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1399/2783] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:27 2022] PUT /identity/v3/projects/c5218a711059454987d15a74f1effcdf/users/f49107b1627e48dd929016de7f93576b/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:27.440547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a4c9b-b5cc-42db-ac50-02f1590a4c29 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:27.440840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a4c9b-b5cc-42db-ac50-02f1590a4c29 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:27.441116 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a4c9b-b5cc-42db-ac50-02f1590a4c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:27.460215 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-809a4c9b-b5cc-42db-ac50-02f1590a4c29 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:27.461524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-809a4c9b-b5cc-42db-ac50-02f1590a4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:27.519547 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1385/2784] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:27.529099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:27.530791 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:27.587274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148dfa-0526-4057-9343-e532c1b6bbda admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:27.590865 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:27.591474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:27.591756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:27.592031 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:27.602013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:27.603485 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:27.612866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1400/2785] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:28.728901 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:28.729914 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:28.730403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:28.761787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] MFA Rules not processed for user `f49107b1627e48dd929016de7f93576b`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:28.765970 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:28.828947 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1386/2786] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:28.833229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:28.843905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971c74e-6c43-4c47-a353-1a0d9de05bf8 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:28.848570 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:28.857164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:28.857164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:28.857164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:28.857164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:28.857164 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:28.938250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1401/2787] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:28 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:28.939381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:28.950919 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba0de2-f070-4066-bc78-86ac9c451d6c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:29.013495 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:29.014103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:29.014381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:29.014656 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:29.015315 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:29.017291 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:29.096819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:29.102627 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1387/2788] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:28 2022] POST /identity/v3/users => generated 308 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:55:29.106798 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:29.117985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e1d88-0286-4aed-be42-ccf8f66b83fc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:29.122905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:29.123516 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:29.123808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:29.124134 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:29.124873 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:29.126526 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:29.138012 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1402/2789] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:29.142183 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:29.152171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54523c1-383e-4b0f-843e-f4a0f9bdbb2b admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:29.156778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:29.157472 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:29.157759 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:29.158038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] PATH_INFO: `/v3/projects/ce7c16deffb84b50b940593fbdc78fa0/users/8660d66999384567a2a37cafff865d8a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:29.173176 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7c16deffb84b50b940593fbdc78fa0, user_id=8660d66999384567a2a37cafff865d8a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:29.175888 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:29.247335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1388/2790] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:29 2022] PUT /identity/v3/projects/ce7c16deffb84b50b940593fbdc78fa0/users/8660d66999384567a2a37cafff865d8a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:55:29.251238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:29.262215 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb6388-579a-44a2-9dda-68fd6b621d17 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:29.318438 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd6ab8-6244-4ad0-a3e3-c28b00424300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:29.319003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd6ab8-6244-4ad0-a3e3-c28b00424300 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:29.319282 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd6ab8-6244-4ad0-a3e3-c28b00424300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:29.319556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd6ab8-6244-4ad0-a3e3-c28b00424300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:29.320275 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd6ab8-6244-4ad0-a3e3-c28b00424300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:29.321995 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd6ab8-6244-4ad0-a3e3-c28b00424300 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:29.331595 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1403/2791] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:29.335906 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:29.346088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdfc18-a3ab-42dd-be6e-6289f5a64cdc admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:29.351441 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:29.352095 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:29.352377 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:29.352651 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] PATH_INFO: `/v3/projects/ce7c16deffb84b50b940593fbdc78fa0/users/8660d66999384567a2a37cafff865d8a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:29.354683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7c16deffb84b50b940593fbdc78fa0, user_id=8660d66999384567a2a37cafff865d8a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:29.357370 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:29.368123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1389/2792] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:29 2022] PUT /identity/v3/projects/ce7c16deffb84b50b940593fbdc78fa0/users/8660d66999384567a2a37cafff865d8a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:55:30.613681 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:30.614557 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:30.625326 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:30.643990 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] MFA Rules not processed for user `8660d66999384567a2a37cafff865d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:30.645245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:30.658794 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:30.716348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1404/2793] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:32.006936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:32.069692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739c56f-d3c2-4cf4-9d9c-21e2724e122e admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:32.074290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:32.074920 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:32.075203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:32.075480 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] PATH_INFO: `/v3/users/f49107b1627e48dd929016de7f93576b` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:32.076628 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f49107b1627e48dd929016de7f93576b)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:32.078625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:32.261560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f49107b1627e48dd929016de7f93576b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.262217 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f49107b1627e48dd929016de7f93576b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.262822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f49107b1627e48dd929016de7f93576b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.276877 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f49107b1627e48dd929016de7f93576b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.349085 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f49107b1627e48dd929016de7f93576b'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.361907 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1390/2794] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:32 2022] DELETE /identity/v3/users/f49107b1627e48dd929016de7f93576b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:32.624234 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:32.635901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f60863-3b75-4b7a-adeb-190089caa228 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:32.695423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:32.696060 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:32.696338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:32.696607 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] PATH_INFO: `/v3/projects/c5218a711059454987d15a74f1effcdf` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:32.697776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5218a711059454987d15a74f1effcdf)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:32.699761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:32.821776 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5218a711059454987d15a74f1effcdf', 'request_id': 'req-1dff9ca3-12c8-4eb3-97e1-e00664669847'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.827916 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5218a711059454987d15a74f1effcdf', 'request_id': 'req-1dff9ca3-12c8-4eb3-97e1-e00664669847'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.896355 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5218a711059454987d15a74f1effcdf', 'request_id': 'req-1dff9ca3-12c8-4eb3-97e1-e00664669847'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.907917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] The token cache is being invalidate because project c5218a711059454987d15a74f1effcdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:32.908199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.908817 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:32.910333 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1405/2795] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:32 2022] DELETE /identity/v3/projects/c5218a711059454987d15a74f1effcdf => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:32.915524 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:32.916911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:32.930121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be366c0e-264a-4573-9634-f3883eef9c66 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:32.968319 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:32.987467 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:32.988102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:32.988381 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:32.988653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] PATH_INFO: `/v3/users/8660d66999384567a2a37cafff865d8a` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:32.989838 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8660d66999384567a2a37cafff865d8a)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:32.992242 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:33.255849 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8660d66999384567a2a37cafff865d8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.256581 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8660d66999384567a2a37cafff865d8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.257118 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8660d66999384567a2a37cafff865d8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.268778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8660d66999384567a2a37cafff865d8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.326237 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3fe06b0d-c052-43c4-8738-d2dcbfc7ce95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8660d66999384567a2a37cafff865d8a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.339147 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1391/2796] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:32 2022] DELETE /identity/v3/users/8660d66999384567a2a37cafff865d8a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:33.570445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:33.570445 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:33.582199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff9ca3-12c8-4eb3-97e1-e00664669847 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:33.623815 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:33.640347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:33.640989 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:33.641433 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:33.641713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] PATH_INFO: `/v3/projects/ce7c16deffb84b50b940593fbdc78fa0` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:33.642867 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7c16deffb84b50b940593fbdc78fa0)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:33.644784 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:33.749628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7c16deffb84b50b940593fbdc78fa0', 'request_id': 'req-bcb6764e-a116-4852-8bad-766efdd6c1d1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.755764 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7c16deffb84b50b940593fbdc78fa0', 'request_id': 'req-bcb6764e-a116-4852-8bad-766efdd6c1d1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.811597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7c16deffb84b50b940593fbdc78fa0', 'request_id': 'req-bcb6764e-a116-4852-8bad-766efdd6c1d1'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.822169 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] The token cache is being invalidate because project ce7c16deffb84b50b940593fbdc78fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:33.824576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.824576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-bcb6764e-a116-4852-8bad-766efdd6c1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:33.824576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1406/2797] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:33 2022] DELETE /identity/v3/projects/ce7c16deffb84b50b940593fbdc78fa0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:33.829623 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:33.829915 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:33.830246 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:33.849383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:33.851423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:33.910189 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1392/2798] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:33.919209 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:33.919491 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:33.919763 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:33.940278 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:33.941518 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:33.949750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1407/2799] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:33.953985 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:33.963594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776422d-c3fb-4d75-9785-d6f496875697 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:33.968090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:33.968689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:33.968968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:33.969263 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] PATH_INFO: `/v3/domains` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:33.969946 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80714) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:55:33.970245 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:33.971840 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:33.983905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1393/2800] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 03:55:33.988260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:33.998775 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6a73-2b8c-410f-9e3b-268a15cecb42 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.003755 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.005127 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.005792 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.006081 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.006753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.008808 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:34.116225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1408/2801] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:33 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:34.120909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:34.130847 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2331b26-b52e-4c95-b877-3f04d313c459 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.181179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:34.199060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.200009 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.200182 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.200454 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.201320 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.203783 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:34.389995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1394/2802] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:34 2022] POST /identity/v3/users => generated 286 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:34.394020 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:34.405306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c5ef7-8216-43b5-b009-41c2d9ab3501 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.409807 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.410450 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.410728 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.410993 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.411758 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.413311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:34.424517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1409/2803] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:34.428604 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:34.439034 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3053f-fc75-4662-baa1-a00da53d53a3 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.443564 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b947ed-0d81-4e04-b719-8ceb7b089e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.448809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b947ed-0d81-4e04-b719-8ceb7b089e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.448809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b947ed-0d81-4e04-b719-8ceb7b089e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.448809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b947ed-0d81-4e04-b719-8ceb7b089e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.448809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b947ed-0d81-4e04-b719-8ceb7b089e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.448809 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b947ed-0d81-4e04-b719-8ceb7b089e48 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:34.457880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1395/2804] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:55:34.461924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:34.472712 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1a716-40f1-4ee5-93a4-98fd6162b758 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.477477 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.478205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.478976 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.479258 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] PATH_INFO: `/v3/projects/c437be319e7341d3bb5eb66c78680b2c/users/1114e21a4e724fe1a3ec46b9a375c9a6/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.493939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c437be319e7341d3bb5eb66c78680b2c, user_id=1114e21a4e724fe1a3ec46b9a375c9a6, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.497345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:34.572588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1410/2805] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:34 2022] PUT /identity/v3/projects/c437be319e7341d3bb5eb66c78680b2c/users/1114e21a4e724fe1a3ec46b9a375c9a6/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:34.584208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.584498 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.584776 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.613943 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] MFA Rules not processed for user `1114e21a4e724fe1a3ec46b9a375c9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:34.615175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:34.680937 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1396/2806] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:34.686411 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:34.747521 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0cc10-f8fb-471b-ae20-79bbb38b8a66 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.752736 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.753380 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.753662 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.753964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.754635 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.756535 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:34.770439 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:34.842503 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1411/2807] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:34 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:34.846733 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:34.858209 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf8487-20b6-4605-b84b-80e09ade9312 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:34.913633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:34.914256 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:34.914541 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:34.914851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] PATH_INFO: `/v3/users` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:34.915528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:34.918208 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:35.011145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1397/2808] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:34 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 03:55:35.015311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:35.026092 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb4bf2-b267-49f1-97bc-ae24452bc6b3 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:35.030637 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:35.031233 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:35.031547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:35.031822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:35.032534 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:35.034158 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:35.045527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1412/2809] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:35.049569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:35.059637 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924152-cd31-43fb-b24b-50c633cd5e5c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:35.064238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:35.064893 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:35.065179 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:35.065489 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] PATH_INFO: `/v3/projects/6b51f30f8a4f44ee8ab47c523ccd1dfa/users/726aef8a8a8c433bb2c90c457cfd5fca/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:35.080139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b51f30f8a4f44ee8ab47c523ccd1dfa, user_id=726aef8a8a8c433bb2c90c457cfd5fca, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:35.082859 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:35.154417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1398/2810] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:35 2022] PUT /identity/v3/projects/6b51f30f8a4f44ee8ab47c523ccd1dfa/users/726aef8a8a8c433bb2c90c457cfd5fca/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:35.158394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:35.169304 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19051092-aa97-442f-b4d4-de9d8bd05506 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:35.226269 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e293d-d4f4-46e5-bc09-b20d3fe8a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:35.226805 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e293d-d4f4-46e5-bc09-b20d3fe8a6de admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:35.227087 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e293d-d4f4-46e5-bc09-b20d3fe8a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:35.227362 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e293d-d4f4-46e5-bc09-b20d3fe8a6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:35.228079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e293d-d4f4-46e5-bc09-b20d3fe8a6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:35.230444 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e293d-d4f4-46e5-bc09-b20d3fe8a6de admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:35.239964 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1413/2811] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:35.245153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:35.255570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66561e-7776-4735-980a-90b3bb5c1485 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:35.260619 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:35.261310 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:35.261594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:35.261869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] PATH_INFO: `/v3/projects/6b51f30f8a4f44ee8ab47c523ccd1dfa/users/726aef8a8a8c433bb2c90c457cfd5fca/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:35.263882 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b51f30f8a4f44ee8ab47c523ccd1dfa, user_id=726aef8a8a8c433bb2c90c457cfd5fca, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:35.266562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:35.276692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1399/2812] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:35 2022] PUT /identity/v3/projects/6b51f30f8a4f44ee8ab47c523ccd1dfa/users/726aef8a8a8c433bb2c90c457cfd5fca/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:35.286727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:35.287013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:35.287306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:35.315494 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] MFA Rules not processed for user `726aef8a8a8c433bb2c90c457cfd5fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:35.316693 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:35.383836 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1414/2813] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:35.396168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:35.397921 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:35.408574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:35.456397 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d39e07-6efd-4690-a9a1-0a954a9deebb admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:35.459980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e25ea2-fa36-45b4-a95e-56fa78a977e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:35.460579 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25ea2-fa36-45b4-a95e-56fa78a977e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:35.460854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25ea2-fa36-45b4-a95e-56fa78a977e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:35.461120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e25ea2-fa36-45b4-a95e-56fa78a977e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:35.470812 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e25ea2-fa36-45b4-a95e-56fa78a977e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:35.472238 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e25ea2-fa36-45b4-a95e-56fa78a977e0 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:35.482035 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1400/2814] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:36.503175 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:36.566692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c30305-a6f8-4279-bffb-bb66c05f95c2 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:36.571520 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:36.572305 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:36.572588 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:36.572892 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] PATH_INFO: `/v3/users/1114e21a4e724fe1a3ec46b9a375c9a6` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:36.574112 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1114e21a4e724fe1a3ec46b9a375c9a6)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:36.576124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:36.580685 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:36.766952 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1114e21a4e724fe1a3ec46b9a375c9a6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:36.773506 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1114e21a4e724fe1a3ec46b9a375c9a6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:36.774163 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1114e21a4e724fe1a3ec46b9a375c9a6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:36.786889 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1114e21a4e724fe1a3ec46b9a375c9a6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:36.879136 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1114e21a4e724fe1a3ec46b9a375c9a6'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:36.886053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1415/2815] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:36 2022] DELETE /identity/v3/users/1114e21a4e724fe1a3ec46b9a375c9a6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:36.890617 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:36.891132 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:36.891278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:36.910771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:36.911967 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:36.969644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1401/2816] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:55:36.979811 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:36.985155 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:37.041301 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7071db-edad-4043-99bc-c5fb67daf0cc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:37.044711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:37.045357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:37.045636 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:37.045904 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:37.054707 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:37.056135 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:37.066180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1416/2817] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:37.436953 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:37.438390 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:37.455284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b56a-8236-4f72-a9d6-050c1fe8b9b7 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:37.460204 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:37.460854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:37.461191 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:37.461570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] PATH_INFO: `/v3/projects/c437be319e7341d3bb5eb66c78680b2c` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:37.463367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c437be319e7341d3bb5eb66c78680b2c)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:37.465225 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:37.590490 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c437be319e7341d3bb5eb66c78680b2c', 'request_id': 'req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.596570 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c437be319e7341d3bb5eb66c78680b2c', 'request_id': 'req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.667767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c437be319e7341d3bb5eb66c78680b2c', 'request_id': 'req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.678719 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] The token cache is being invalidate because project c437be319e7341d3bb5eb66c78680b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:37.684635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.684635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.684635 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1402/2818] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:37 2022] DELETE /identity/v3/projects/c437be319e7341d3bb5eb66c78680b2c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:37.685654 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:37.695423 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4a944-3487-4d4a-a28f-118f1e5cb7cb service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:37.750913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:37.751546 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:37.751827 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:37.752101 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] PATH_INFO: `/v3/users/726aef8a8a8c433bb2c90c457cfd5fca` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:37.753285 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=726aef8a8a8c433bb2c90c457cfd5fca)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:37.755299 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:37.841016 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:37.991156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '726aef8a8a8c433bb2c90c457cfd5fca'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.998674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '726aef8a8a8c433bb2c90c457cfd5fca'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:37.999351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '726aef8a8a8c433bb2c90c457cfd5fca'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.012648 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '726aef8a8a8c433bb2c90c457cfd5fca'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.080358 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-375c108e-0c09-4072-857a-36ba3a3c6f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '726aef8a8a8c433bb2c90c457cfd5fca'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.086787 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1417/2819] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:37 2022] DELETE /identity/v3/users/726aef8a8a8c433bb2c90c457cfd5fca => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:38.465726 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:38.467283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:38.481625 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9915a-d7eb-4d6e-adbc-a2908771a88a admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:38.512854 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:38.540447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:38.541111 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:38.541420 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:38.541697 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] PATH_INFO: `/v3/projects/6b51f30f8a4f44ee8ab47c523ccd1dfa` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:38.542910 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b51f30f8a4f44ee8ab47c523ccd1dfa)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:38.544843 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:38.651660 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b51f30f8a4f44ee8ab47c523ccd1dfa', 'request_id': 'req-043e6514-eecc-440d-8316-706218cf8004'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.657744 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b51f30f8a4f44ee8ab47c523ccd1dfa', 'request_id': 'req-043e6514-eecc-440d-8316-706218cf8004'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.720088 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b51f30f8a4f44ee8ab47c523ccd1dfa', 'request_id': 'req-043e6514-eecc-440d-8316-706218cf8004'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.731484 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] The token cache is being invalidate because project 6b51f30f8a4f44ee8ab47c523ccd1dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:38.732876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.732876 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-043e6514-eecc-440d-8316-706218cf8004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:38.733911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1403/2820] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:38 2022] DELETE /identity/v3/projects/6b51f30f8a4f44ee8ab47c523ccd1dfa => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:38.739823 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:38.740311 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:38.740441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:38.762357 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:38.763563 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:38.821453 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1418/2821] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:38.883507 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:38.883797 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:38.884098 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:38.904399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:38.905642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:38.913835 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1404/2822] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:38.918768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:38.928379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98d3a4-fda3-4829-a59c-e95b7883074e None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:38.933639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:38.934237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:38.934519 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:38.934789 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:38.935482 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:55:38.935783 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:38.937418 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:38.948971 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1419/2823] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:55:38.953171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:38.962447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f06a3-16a2-4e9a-86a1-d3591ea252ab None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:38.967391 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:38.967995 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:38.968279 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:38.968555 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:38.969284 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:38.971290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.059415 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1405/2824] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:38 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:39.063766 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.074214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428e44d-20ee-4daf-9211-934701272e0c admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.130622 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.131245 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.131507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.131779 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.132446 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.134409 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.240838 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1420/2825] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:39 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:39.245614 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.256070 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d883099b-6c3a-4feb-8d93-05284ea490c9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.260715 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.261346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.261632 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.261902 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.262593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.264184 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.274053 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1406/2826] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:39.278529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.289056 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf53f7-24b6-41a4-b3c1-b21475ccecab admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.294499 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29a3ee-5129-4291-a3aa-6af4d447596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.295140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a3ee-5129-4291-a3aa-6af4d447596e admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.295420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a3ee-5129-4291-a3aa-6af4d447596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.295692 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a3ee-5129-4291-a3aa-6af4d447596e admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.296394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29a3ee-5129-4291-a3aa-6af4d447596e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.298030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29a3ee-5129-4291-a3aa-6af4d447596e admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.308572 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1421/2827] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:55:39.312728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.322760 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a845e21-1d1d-4547-bfea-0a8bd5abde7c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.327292 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.327936 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.328216 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.328491 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] PATH_INFO: `/v3/projects/e5ab6515e4c140daa8be5b3811558281/users/d1ab56fce8f04e64b7c19391cc85325c/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.344037 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ab6515e4c140daa8be5b3811558281, user_id=d1ab56fce8f04e64b7c19391cc85325c, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.346725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.434927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1407/2828] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:39 2022] PUT /identity/v3/projects/e5ab6515e4c140daa8be5b3811558281/users/d1ab56fce8f04e64b7c19391cc85325c/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:39.445108 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.445434 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.445726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.467259 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:39.474395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] MFA Rules not processed for user `d1ab56fce8f04e64b7c19391cc85325c`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:39.475852 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:39.541583 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1422/2829] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:39.545981 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.608170 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6482b-53af-4a97-9674-1e03158aa0b9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.612822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.616510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.616510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.616510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.616510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.616510 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.622055 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:39.687405 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1408/2830] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:39 2022] POST /identity/v3/projects => generated 393 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:39.691492 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.702896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c3506-f4fd-4a0e-8524-aa6ef1ca198d None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.757924 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.758585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.758796 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.759066 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.760394 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.762347 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.866864 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1423/2831] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:39 2022] POST /identity/v3/users => generated 303 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:39.870890 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.881594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502c4f4-6c94-4e4d-a332-14b0f49ba526 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.887139 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.894533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.894533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.894533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.894533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.894533 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:39.901551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1409/2832] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:39.905688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:39.915575 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd0518-df62-4812-b4f7-0fd0eb84d62f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:39.920384 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:39.921027 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:39.921345 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:39.921618 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] PATH_INFO: `/v3/projects/028ef641ffc54826b33b30cfa13b6d09/users/300a6e1cedbc408cb05ed4c6b6c811ba/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:39.936555 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=028ef641ffc54826b33b30cfa13b6d09, user_id=300a6e1cedbc408cb05ed4c6b6c811ba, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:39.939236 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:40.004901 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1424/2833] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:39 2022] PUT /identity/v3/projects/028ef641ffc54826b33b30cfa13b6d09/users/300a6e1cedbc408cb05ed4c6b6c811ba/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:40.008975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:40.020138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5f88c-c51a-4f85-a73d-7f6bd7cff7ae admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:40.077283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d8034a-5565-49b1-a253-6e14be2b954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:40.077924 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8034a-5565-49b1-a253-6e14be2b954b admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:40.078196 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8034a-5565-49b1-a253-6e14be2b954b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:40.078473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d8034a-5565-49b1-a253-6e14be2b954b admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:40.079262 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8034a-5565-49b1-a253-6e14be2b954b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:40.080867 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d8034a-5565-49b1-a253-6e14be2b954b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:40.090805 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1410/2834] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:40.095103 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:40.105937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73c0b4-224e-4528-b3f3-8da332423e85 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:40.110539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:40.111192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:40.111472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:40.111745 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] PATH_INFO: `/v3/projects/028ef641ffc54826b33b30cfa13b6d09/users/300a6e1cedbc408cb05ed4c6b6c811ba/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:40.113701 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=028ef641ffc54826b33b30cfa13b6d09, user_id=300a6e1cedbc408cb05ed4c6b6c811ba, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:40.116351 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:40.127170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1425/2835] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:40 2022] PUT /identity/v3/projects/028ef641ffc54826b33b30cfa13b6d09/users/300a6e1cedbc408cb05ed4c6b6c811ba/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:40.137965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:40.138258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:40.138545 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:40.165889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] MFA Rules not processed for user `300a6e1cedbc408cb05ed4c6b6c811ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:40.167270 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:40.235010 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1411/2836] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:40.246165 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:40.247802 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:40.304441 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47022845-e6ef-4ea6-a2f8-a9f0f967ae7a admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:40.309077 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:40.309726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:40.309998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:40.310272 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:40.317424 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:40.319790 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:40.321257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:40.332425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1426/2837] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:40.380287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:40.381639 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:40.394251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434547c4-3c2f-4951-862e-8ae095018218 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:40.397811 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664c95c-5fdf-4fa4-be8e-abf669be3cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:40.398406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664c95c-5fdf-4fa4-be8e-abf669be3cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:40.398689 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664c95c-5fdf-4fa4-be8e-abf669be3cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:40.398963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664c95c-5fdf-4fa4-be8e-abf669be3cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:40.451663 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:40.455289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664c95c-5fdf-4fa4-be8e-abf669be3cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:40.456737 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664c95c-5fdf-4fa4-be8e-abf669be3cf3 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:40.465531 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1412/2838] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:45.020594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:45.082556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bcff2-de51-40b9-9d88-23352a1405f6 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:45.087096 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:45.087727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:45.088010 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:45.088284 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] PATH_INFO: `/v3/users/d1ab56fce8f04e64b7c19391cc85325c` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:45.089640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ab56fce8f04e64b7c19391cc85325c)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:45.091594 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:45.282756 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ab56fce8f04e64b7c19391cc85325c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:45.289480 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ab56fce8f04e64b7c19391cc85325c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:45.290143 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ab56fce8f04e64b7c19391cc85325c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:45.302389 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ab56fce8f04e64b7c19391cc85325c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:45.371761 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ab56fce8f04e64b7c19391cc85325c'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:45.377972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1427/2839] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:45 2022] DELETE /identity/v3/users/d1ab56fce8f04e64b7c19391cc85325c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:45.382482 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:45.382944 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:45.383045 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:45.402022 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:45.403278 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:45.459488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1413/2840] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:45.470026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:45.471428 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:45.528705 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8c855-f912-4c03-b625-bf74291ec47b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:45.532379 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:45.533079 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:45.533335 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:45.533610 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:45.543229 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:45.544669 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:45.554068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1428/2841] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:55:45.979520 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:45.981825 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:45.997334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b05e85-32d8-41fa-b82d-7792db6bb312 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:46.002180 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:46.002851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:46.003156 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:46.007991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] PATH_INFO: `/v3/projects/e5ab6515e4c140daa8be5b3811558281` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:46.007991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ab6515e4c140daa8be5b3811558281)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:46.007991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:46.131500 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ab6515e4c140daa8be5b3811558281', 'request_id': 'req-90dbba56-dbe8-4252-9694-89aa44b72628'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.137779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ab6515e4c140daa8be5b3811558281', 'request_id': 'req-90dbba56-dbe8-4252-9694-89aa44b72628'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.211775 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ab6515e4c140daa8be5b3811558281', 'request_id': 'req-90dbba56-dbe8-4252-9694-89aa44b72628'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.223504 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] The token cache is being invalidate because project e5ab6515e4c140daa8be5b3811558281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:46.223787 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.224402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.226200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1414/2842] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:45 2022] DELETE /identity/v3/projects/e5ab6515e4c140daa8be5b3811558281 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:46.231241 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:46.240959 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e713ee9-47bf-4ae4-a80f-eeacc295747d service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:46.253260 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:46.297238 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:46.297877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:46.298156 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:46.298426 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] PATH_INFO: `/v3/users/300a6e1cedbc408cb05ed4c6b6c811ba` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:46.299603 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=300a6e1cedbc408cb05ed4c6b6c811ba)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:46.301556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:46.528475 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300a6e1cedbc408cb05ed4c6b6c811ba'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.535124 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300a6e1cedbc408cb05ed4c6b6c811ba'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.535866 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300a6e1cedbc408cb05ed4c6b6c811ba'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.547578 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300a6e1cedbc408cb05ed4c6b6c811ba'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.633331 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-396b38f7-118b-41ae-8494-a12296a33c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300a6e1cedbc408cb05ed4c6b6c811ba'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:46.640191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1429/2843] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:46 2022] DELETE /identity/v3/users/300a6e1cedbc408cb05ed4c6b6c811ba => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:47.059417 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.060714 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:47.074316 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbba56-dbe8-4252-9694-89aa44b72628 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.132252 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.132884 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.133163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.133460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] PATH_INFO: `/v3/projects/028ef641ffc54826b33b30cfa13b6d09` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.134599 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=028ef641ffc54826b33b30cfa13b6d09)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.137150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.158258 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:47.251622 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '028ef641ffc54826b33b30cfa13b6d09', 'request_id': 'req-5f8b3594-51f3-4284-96b8-f8b101eecb1c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:47.257732 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '028ef641ffc54826b33b30cfa13b6d09', 'request_id': 'req-5f8b3594-51f3-4284-96b8-f8b101eecb1c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:47.325905 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '028ef641ffc54826b33b30cfa13b6d09', 'request_id': 'req-5f8b3594-51f3-4284-96b8-f8b101eecb1c'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:47.336789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] The token cache is being invalidate because project 028ef641ffc54826b33b30cfa13b6d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:47.337233 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:47.337869 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-5f8b3594-51f3-4284-96b8-f8b101eecb1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:47.340556 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1415/2844] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:47 2022] DELETE /identity/v3/projects/028ef641ffc54826b33b30cfa13b6d09 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:47.345624 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.346201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.346201 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.365390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:47.366576 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:47.425984 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1430/2845] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:47.435259 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.435546 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.435823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.457090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:47.458312 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:47.467067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1416/2846] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:47.471204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.480848 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54afe836-d35d-4b42-87fd-a4985f90aa2b None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.485338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.485938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.486216 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.486483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.487167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:55:47.487466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.489068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.500649 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1431/2847] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:55:47.504927 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.514736 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f82c1-b924-4933-baa1-5df51da02823 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.519443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749543ab-2733-4687-a322-646495d401cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.520044 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749543ab-2733-4687-a322-646495d401cf admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.520322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749543ab-2733-4687-a322-646495d401cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.520593 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749543ab-2733-4687-a322-646495d401cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.521283 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749543ab-2733-4687-a322-646495d401cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.523203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749543ab-2733-4687-a322-646495d401cf admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.601762 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1417/2848] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:47 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:47.613554 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.623600 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ed0d0-ccbf-40e6-9bbf-1122cb07781f admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.643274 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:47.679360 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.679966 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.680247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.680522 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.681191 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.683179 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.789951 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1432/2849] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:47 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:47.794633 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749543ab-2733-4687-a322-646495d401cf admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.805439 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749543ab-2733-4687-a322-646495d401cf admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.810218 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.810821 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.811102 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.811376 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.812071 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.813707 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.823974 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1418/2850] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:47.827854 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.837727 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c4abd-d9c2-46cc-add2-4825a34aecb8 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.842923 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51ff0b-1c49-4e84-9930-5591a0da9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.848670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51ff0b-1c49-4e84-9930-5591a0da9193 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.848670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51ff0b-1c49-4e84-9930-5591a0da9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.848670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51ff0b-1c49-4e84-9930-5591a0da9193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.848670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51ff0b-1c49-4e84-9930-5591a0da9193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.848670 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51ff0b-1c49-4e84-9930-5591a0da9193 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.857846 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1433/2851] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:47.862050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:47.871975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e5a6-4f3a-48a7-9600-2d6c7d9507a5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:47.876475 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:47.877121 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.877430 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.877701 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] PATH_INFO: `/v3/projects/87cf33679ae54fcb8e3a5e187bcfc3f8/users/64222459c7654bc593da1077f220db8f/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:47.892406 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87cf33679ae54fcb8e3a5e187bcfc3f8, user_id=64222459c7654bc593da1077f220db8f, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:47.895103 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:47.977304 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1419/2852] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:47 2022] PUT /identity/v3/projects/87cf33679ae54fcb8e3a5e187bcfc3f8/users/64222459c7654bc593da1077f220db8f/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:47.987911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:47.988198 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:47.988469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.015998 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] MFA Rules not processed for user `64222459c7654bc593da1077f220db8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:48.017224 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:48.080769 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1434/2853] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:48.085264 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.146649 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ee2b2-c243-4c8a-82f6-a3728de69f8c admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.151157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.152001 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.152448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.152725 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.153410 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.155323 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.236228 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1420/2854] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:48 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:48.240532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.252131 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b37eb-43be-4b26-a49e-e1e7d2fe65e5 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.307978 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.308597 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.308876 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.309152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.309845 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.311773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.410327 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1435/2855] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:48 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:48.414396 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.426060 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b66bb8-0d9b-4238-a49e-01899142cc67 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.430551 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.431309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.431709 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.431983 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.432975 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.435163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.445968 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1421/2856] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:48.450488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.460395 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1479cb5-0938-4d2a-b57d-40d5f0932239 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.465545 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.466192 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.466469 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.466741 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] PATH_INFO: `/v3/projects/d3049bcab8f041149dbdd171e9e417a3/users/56b1b06745e8404bb396831b1dad4d1a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.481985 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3049bcab8f041149dbdd171e9e417a3, user_id=56b1b06745e8404bb396831b1dad4d1a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.484632 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.549253 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:48.554753 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1436/2857] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:48 2022] PUT /identity/v3/projects/d3049bcab8f041149dbdd171e9e417a3/users/56b1b06745e8404bb396831b1dad4d1a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:55:48.559153 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.569163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b12045-6e9d-45ed-9637-86da28c22c43 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.581611 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:48.626159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68500b0a-0c0e-42e3-a64c-726ec8ce088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.626771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500b0a-0c0e-42e3-a64c-726ec8ce088b admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.627052 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500b0a-0c0e-42e3-a64c-726ec8ce088b admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.627327 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68500b0a-0c0e-42e3-a64c-726ec8ce088b admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.628050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68500b0a-0c0e-42e3-a64c-726ec8ce088b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.629810 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68500b0a-0c0e-42e3-a64c-726ec8ce088b admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.640087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1422/2858] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:48.644083 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.655298 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e30886-886c-48f0-9891-fe0086969814 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.660172 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.660884 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.661113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.661417 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] PATH_INFO: `/v3/projects/d3049bcab8f041149dbdd171e9e417a3/users/56b1b06745e8404bb396831b1dad4d1a/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.663376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3049bcab8f041149dbdd171e9e417a3, user_id=56b1b06745e8404bb396831b1dad4d1a, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.666102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.676961 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1437/2859] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:48 2022] PUT /identity/v3/projects/d3049bcab8f041149dbdd171e9e417a3/users/56b1b06745e8404bb396831b1dad4d1a/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:48.688155 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.688447 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.688734 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.718293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] MFA Rules not processed for user `56b1b06745e8404bb396831b1dad4d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:48.720335 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:48.787820 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1423/2860] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:48.799071 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:48.800813 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:48.860899 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ec7a7-0956-4920-abcf-26e6ce9cec85 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:48.864543 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:48.865152 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:48.865458 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:48.865726 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:48.874773 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:48.876205 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:48.885599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1438/2861] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:49.106000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:49.115676 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3db0d3-5e97-4247-b2ec-aead463914cd None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:49.119305 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f1f3c-786c-4eb5-a230-e4bb94cea171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:49.119911 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f1f3c-786c-4eb5-a230-e4bb94cea171 service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:49.120704 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f1f3c-786c-4eb5-a230-e4bb94cea171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:49.120980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f1f3c-786c-4eb5-a230-e4bb94cea171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:49.177543 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f1f3c-786c-4eb5-a230-e4bb94cea171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:49.179043 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f1f3c-786c-4eb5-a230-e4bb94cea171 service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:49.188631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1424/2862] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 03:55:55.519529 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:55.588210 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f90879-641f-4cf8-a31e-46b19b9a431b service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:55.593819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:55.594466 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:55.594750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:55.595026 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] PATH_INFO: `/v3/users/64222459c7654bc593da1077f220db8f` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:55.596287 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64222459c7654bc593da1077f220db8f)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:55.599045 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:55.806760 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64222459c7654bc593da1077f220db8f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:55.814704 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64222459c7654bc593da1077f220db8f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:55.815397 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64222459c7654bc593da1077f220db8f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:55.828541 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64222459c7654bc593da1077f220db8f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:55.931941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64222459c7654bc593da1077f220db8f'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:55.939030 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1439/2863] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:55 2022] DELETE /identity/v3/users/64222459c7654bc593da1077f220db8f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:55.943819 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:55.946289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:55.946289 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:55.964657 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:55.966434 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:56.008286 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:56.037528 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1425/2864] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:56.049217 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:56.050709 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:56.063276 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:56.120711 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118ec9-38a3-41db-8ba0-ec85012a2ba5 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:56.124910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:56.125548 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:56.125826 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:56.126094 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:56.135425 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:56.137713 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:56.147376 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1440/2865] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:56.561815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:56.563294 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:56.575465 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15ae6e-d450-4165-aaf2-e3a18e1aff83 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:56.580050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:56.580693 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:56.580982 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:56.581288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] PATH_INFO: `/v3/projects/87cf33679ae54fcb8e3a5e187bcfc3f8` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:56.582446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87cf33679ae54fcb8e3a5e187bcfc3f8)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:56.584369 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:56.687203 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87cf33679ae54fcb8e3a5e187bcfc3f8', 'request_id': 'req-352596ab-bb87-4521-84be-5e80bb324ba7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.693330 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87cf33679ae54fcb8e3a5e187bcfc3f8', 'request_id': 'req-352596ab-bb87-4521-84be-5e80bb324ba7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.753959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87cf33679ae54fcb8e3a5e187bcfc3f8', 'request_id': 'req-352596ab-bb87-4521-84be-5e80bb324ba7'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.764519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] The token cache is being invalidate because project 87cf33679ae54fcb8e3a5e187bcfc3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:56.764804 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.765443 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.766942 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1426/2866] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:56 2022] DELETE /identity/v3/projects/87cf33679ae54fcb8e3a5e187bcfc3f8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:56.770929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:56.780841 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf43-35dd-48f4-8916-071078b7f30e service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:56.838797 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:56.839390 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:56.839673 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:56.840696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] PATH_INFO: `/v3/users/56b1b06745e8404bb396831b1dad4d1a` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:56.842058 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b1b06745e8404bb396831b1dad4d1a)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:56.843994 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:56.985015 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b1b06745e8404bb396831b1dad4d1a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.992167 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b1b06745e8404bb396831b1dad4d1a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:56.992929 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b1b06745e8404bb396831b1dad4d1a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.004875 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b1b06745e8404bb396831b1dad4d1a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.079818 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-432a2228-2e6d-4b30-ae1a-a08be3164795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b1b06745e8404bb396831b1dad4d1a'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.086069 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1441/2867] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:56 2022] DELETE /identity/v3/users/56b1b06745e8404bb396831b1dad4d1a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:57.492631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:57.494093 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:57.507642 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352596ab-bb87-4521-84be-5e80bb324ba7 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:57.567476 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:57.568112 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:57.568392 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:57.568668 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] PATH_INFO: `/v3/projects/d3049bcab8f041149dbdd171e9e417a3` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:57.569863 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3049bcab8f041149dbdd171e9e417a3)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:57.571771 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:57.668827 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3049bcab8f041149dbdd171e9e417a3', 'request_id': 'req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.675030 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3049bcab8f041149dbdd171e9e417a3', 'request_id': 'req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.736728 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3049bcab8f041149dbdd171e9e417a3', 'request_id': 'req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.738824 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:57.747644 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] The token cache is being invalidate because project d3049bcab8f041149dbdd171e9e417a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:55:57.747926 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.748631 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-6aaed7de-721e-4e8e-80e6-f9d7328e8fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:55:57.750168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1427/2868] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:55:57 2022] DELETE /identity/v3/projects/d3049bcab8f041149dbdd171e9e417a3 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:57.755126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:57.755414 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:57.755688 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:57.776194 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:57.778111 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:57.802317 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:57.836497 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1442/2869] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:57.845530 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:57.845823 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:57.846096 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:57.865574 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:57.866766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:57.874749 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1428/2870] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:57.878968 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:57.889731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932a665-6a16-43ed-b9ac-e408ecdda3ee None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:57.894415 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:57.895013 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:57.895288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:57.895628 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:57.896283 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:55:57.896586 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:57.898203 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:57.909963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1443/2871] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:55:57.914146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:57.923792 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f544bb4-02c4-4d3c-afd8-6fa8c68ddeb1 None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:57.928303 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:57.928898 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:57.929177 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:57.929473 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:57.930150 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:57.932038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.047600 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1429/2872] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:57 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:58.051800 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.061909 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4ae34-d6b5-4aff-b0fd-2dd0f34387fd admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.117429 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.118040 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.118324 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.118599 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.119265 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.121539 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.198872 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1444/2873] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:58 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:58.203138 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.213446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a5ff7-dd72-4c38-8430-4ddc7b1dedbc admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.218566 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.219163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.219440 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.219706 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.220387 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.222006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.233057 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1430/2874] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:58.237053 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.247427 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719c99a-2466-4686-ad85-e6005ad49dca admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.252082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a233e-8066-4820-8356-fdef70c6fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.252678 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a233e-8066-4820-8356-fdef70c6fae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.252955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a233e-8066-4820-8356-fdef70c6fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.253257 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a233e-8066-4820-8356-fdef70c6fae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.253955 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a233e-8066-4820-8356-fdef70c6fae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.255547 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a233e-8066-4820-8356-fdef70c6fae1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.266118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1445/2875] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:58.270081 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.280879 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11421b45-5043-48f9-bdef-103adb411c20 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.285477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.286115 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.286422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.286692 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] PATH_INFO: `/v3/projects/72c741c44d994fd3a9545c8c85183a97/users/57671855fd054cf49409eb2c47dc1f38/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.302100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c741c44d994fd3a9545c8c85183a97, user_id=57671855fd054cf49409eb2c47dc1f38, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.304778 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.354120 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1431/2876] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:58 2022] PUT /identity/v3/projects/72c741c44d994fd3a9545c8c85183a97/users/57671855fd054cf49409eb2c47dc1f38/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:58.364887 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.365214 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.365533 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.393671 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] MFA Rules not processed for user `57671855fd054cf49409eb2c47dc1f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:58.394905 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:58.458402 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1446/2877] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:55:58.462979 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.523880 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791893a-7d68-4302-9ee5-8ae631875bf6 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.529383 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.530000 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.530285 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.530562 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.531223 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.533165 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.642039 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1432/2878] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:55:58 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:58.646170 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.658118 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90447fc3-20c7-45f4-b68a-dce626672b38 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.677697 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:58.717348 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.717883 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.718159 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.718456 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.719117 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.721019 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.832715 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1447/2879] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:55:58 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:55:58.836792 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.847423 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e65a89-840b-4681-80bb-f7081c426de9 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.852224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.852828 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.853113 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.853422 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.854151 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.855826 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.866337 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1433/2880] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:55:58.870639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.880910 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34d699-a9c2-4446-9f68-3c7c6af833fc admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:58.885421 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:58.886065 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:58.886341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:58.887332 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] PATH_INFO: `/v3/projects/a53a12c765fd442e8698abf7939d7066/users/3059ba8cdb504408bc12d27d6cda60df/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:58.901945 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53a12c765fd442e8698abf7939d7066, user_id=3059ba8cdb504408bc12d27d6cda60df, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:58.904941 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:58.977338 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1448/2881] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:58 2022] PUT /identity/v3/projects/a53a12c765fd442e8698abf7939d7066/users/3059ba8cdb504408bc12d27d6cda60df/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:58.981212 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:58.991211 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36aa76-14f7-4532-b06f-91ca4325ce78 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:59.047963 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59006c-9848-42f3-be81-0563496de097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:59.048569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59006c-9848-42f3-be81-0563496de097 admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:59.048851 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59006c-9848-42f3-be81-0563496de097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:59.049126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59006c-9848-42f3-be81-0563496de097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:59.049845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59006c-9848-42f3-be81-0563496de097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:59.051460 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c59006c-9848-42f3-be81-0563496de097 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:59.053594 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:55:59.062353 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1434/2882] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 03:55:59.071144 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:59.082180 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe918e-62d2-4db5-b12b-987a7caa7c71 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:59.086937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:59.087585 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] REQUEST_METHOD: `PUT` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:59.087868 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:59.088140 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] PATH_INFO: `/v3/projects/a53a12c765fd442e8698abf7939d7066/users/3059ba8cdb504408bc12d27d6cda60df/roles/2388a99b8be348d1a010f30ed34e35f8` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:59.090250 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a53a12c765fd442e8698abf7939d7066, user_id=3059ba8cdb504408bc12d27d6cda60df, role_id=2388a99b8be348d1a010f30ed34e35f8)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:59.092972 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:59.103584 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1449/2883] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:55:59 2022] PUT /identity/v3/projects/a53a12c765fd442e8698abf7939d7066/users/3059ba8cdb504408bc12d27d6cda60df/roles/2388a99b8be348d1a010f30ed34e35f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:55:59.115789 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:59.116123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:59.116403 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:59.145050 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] MFA Rules not processed for user `3059ba8cdb504408bc12d27d6cda60df`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:55:59.146297 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:59.215432 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1435/2884] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:55:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:55:59.226938 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:59.228639 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:55:59.285690 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc3f6-8e17-4312-af66-62c237ebb897 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:59.289320 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:59.289926 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:59.290204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:59.290474 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:59.300120 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:59.301598 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:59.311527 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1450/2885] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:59.430309 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:55:59.439922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133686af-0285-47e6-ba8b-f29e290f430c None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:55:59.443444 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c6050-ac6f-4b0e-8053-ccfe68e2ce7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:55:59.444097 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6050-ac6f-4b0e-8053-ccfe68e2ce7b service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:55:59.444385 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6050-ac6f-4b0e-8053-ccfe68e2ce7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:55:59.444664 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6050-ac6f-4b0e-8053-ccfe68e2ce7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:55:59.502168 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c6050-ac6f-4b0e-8053-ccfe68e2ce7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:55:59.503549 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c6050-ac6f-4b0e-8053-ccfe68e2ce7b service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:55:59.513005 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1436/2886] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:55:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:55:59.980052 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:00.041956 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e23e-a18d-4ea0-bb38-47a4c6011819 service cinder] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:00.047099 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:00.047738 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:00.048017 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:00.048288 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] PATH_INFO: `/v3/users/57671855fd054cf49409eb2c47dc1f38` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:00.049483 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57671855fd054cf49409eb2c47dc1f38)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:00.051416 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:00.073465 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:00.280556 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57671855fd054cf49409eb2c47dc1f38'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:00.287302 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57671855fd054cf49409eb2c47dc1f38'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:00.287967 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57671855fd054cf49409eb2c47dc1f38'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:00.300878 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57671855fd054cf49409eb2c47dc1f38'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:00.376696 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57671855fd054cf49409eb2c47dc1f38'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:00.383082 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1451/2887] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:55:59 2022] DELETE /identity/v3/users/57671855fd054cf49409eb2c47dc1f38 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:56:00.388158 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:00.388621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:00.388738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:00.408358 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:56:00.409624 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:00.470806 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1437/2888] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:56:00.481350 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:00.482747 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:00.539084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e3b67-c36d-4c4e-9934-6cee8950b213 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:00.542809 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:00.544054 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:00.544336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:00.544630 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:00.553871 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:00.555237 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:00.564565 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1452/2889] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:56:00.961492 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:00.963019 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:00.976486 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff5a83-f91e-4a87-a002-a0a188c01fec None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:00.981598 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:00.982402 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:00.982572 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:00.982831 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] PATH_INFO: `/v3/projects/72c741c44d994fd3a9545c8c85183a97` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:00.984038 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c741c44d994fd3a9545c8c85183a97)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:00.986571 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:01.006221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=100 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:01.103477 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c741c44d994fd3a9545c8c85183a97', 'request_id': 'req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.109540 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c741c44d994fd3a9545c8c85183a97', 'request_id': 'req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.217400 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c741c44d994fd3a9545c8c85183a97', 'request_id': 'req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.228621 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] The token cache is being invalidate because project 72c741c44d994fd3a9545c8c85183a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:56:01.228909 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.229542 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.231159 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1438/2890] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:56:00 2022] DELETE /identity/v3/projects/72c741c44d994fd3a9545c8c85183a97 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:56:01.235336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:01.244917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fadd0-a5ae-49fc-a5bd-97247294a6b4 service neutron] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:01.303188 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:01.303819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:01.304113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:01.304370 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] PATH_INFO: `/v3/users/3059ba8cdb504408bc12d27d6cda60df` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:01.305553 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3059ba8cdb504408bc12d27d6cda60df)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:01.307478 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:01.444877 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:01.447385 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3059ba8cdb504408bc12d27d6cda60df'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.454663 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3059ba8cdb504408bc12d27d6cda60df'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.455322 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3059ba8cdb504408bc12d27d6cda60df'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.467896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3059ba8cdb504408bc12d27d6cda60df'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.546731 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-e17de6c1-7a61-4709-9e4a-1e6ae51ed9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3059ba8cdb504408bc12d27d6cda60df'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:01.552942 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1453/2891] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:56:01 2022] DELETE /identity/v3/users/3059ba8cdb504408bc12d27d6cda60df => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:56:01.996416 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:01.998346 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:02.012626 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387dc45-e1e0-4fad-acf6-8027ad3c9e58 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.068412 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.069100 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.069408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.069680 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] PATH_INFO: `/v3/projects/a53a12c765fd442e8698abf7939d7066` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.070822 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a53a12c765fd442e8698abf7939d7066)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.072791 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.168738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a53a12c765fd442e8698abf7939d7066', 'request_id': 'req-3d842585-c144-4f17-9bbe-09851ed1ff7e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:02.175752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a53a12c765fd442e8698abf7939d7066', 'request_id': 'req-3d842585-c144-4f17-9bbe-09851ed1ff7e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:02.251062 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a53a12c765fd442e8698abf7939d7066', 'request_id': 'req-3d842585-c144-4f17-9bbe-09851ed1ff7e'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:02.264569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] The token cache is being invalidate because project a53a12c765fd442e8698abf7939d7066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:56:02.264569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:02.264569 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-3d842585-c144-4f17-9bbe-09851ed1ff7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:56:02.265020 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1439/2892] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:56:01 2022] DELETE /identity/v3/projects/a53a12c765fd442e8698abf7939d7066 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:56:02.269824 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.270448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.270448 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.290204 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:56:02.291399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:02.350408 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1454/2893] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 03:56:02.358479 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.358769 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.359047 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.378521 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.auth.core [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80714) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:56:02.379738 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:02.388429 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1440/2894] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:56:02.392537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.402102 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd33c8f5-1606-4401-9aaf-5aa267492348 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.407321 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.407913 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.408196 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.408472 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.409168 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 03:56:02.409507 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.411115 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.422537 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1455/2895] 10.222.0.66 () {58 vars in 1153 bytes} [Thu Aug 4 03:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 03:56:02.426767 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.436375 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14907f-9679-4052-b0b9-96fad307d39d None None] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.440959 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.441589 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] REQUEST_METHOD: `POST` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.441872 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.442146 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] PATH_INFO: `/v3/projects` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.442815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.444718 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.544336 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1441/2896] 10.222.0.66 () {60 vars in 1153 bytes} [Thu Aug 4 03:56:02 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:56:02.548488 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.559068 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbf399-f088-4852-be08-c46357875d26 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.615388 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.616076 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.616361 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.616638 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] PATH_INFO: `/v3/users` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.617341 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.619262 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.702420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1456/2897] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Aug 4 03:56:02 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 03:56:02.706640 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.717448 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0a68b-4446-4645-ba9b-2ae353165fac admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.722573 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.723266 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.723536 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.723815 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.724779 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.726426 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.727653 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:02.736992 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1442/2898] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:56:02.742268 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.752983 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7fcbc-4bc6-42b1-b16a-38a1887bf23b admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.757757 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d569c2-b41e-486b-bb08-d771d34078e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.758365 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d569c2-b41e-486b-bb08-d771d34078e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.758645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d569c2-b41e-486b-bb08-d771d34078e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.758921 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d569c2-b41e-486b-bb08-d771d34078e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.759615 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d569c2-b41e-486b-bb08-d771d34078e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.761247 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d569c2-b41e-486b-bb08-d771d34078e1 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.773746 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1457/2899] 10.222.0.66 () {58 vars in 1122 bytes} [Thu Aug 4 03:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 03:56:02.778087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.788171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad542a4d-1006-465e-a884-06a11d294fec admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:02.793333 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:02.793875 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.794161 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.794488 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] PATH_INFO: `/v3/projects/16033dcf8c904cb2af2c8701a4b6a716/users/47c57945a0bb4fc78a812b7d453fca81/roles/e1ef11f06c1e48cba9ed25868e476dc4` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.809288 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16033dcf8c904cb2af2c8701a4b6a716, user_id=47c57945a0bb4fc78a812b7d453fca81, role_id=e1ef11f06c1e48cba9ed25868e476dc4)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:02.812087 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:02.868602 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1443/2900] 10.222.0.66 () {60 vars in 1483 bytes} [Thu Aug 4 03:56:02 2022] PUT /identity/v3/projects/16033dcf8c904cb2af2c8701a4b6a716/users/47c57945a0bb4fc78a812b7d453fca81/roles/e1ef11f06c1e48cba9ed25868e476dc4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:56:02.874558 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fec1cf-bb00-4a1a-82a9-6b6322b93b34 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:02.874849 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fec1cf-bb00-4a1a-82a9-6b6322b93b34 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:02.875125 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fec1cf-bb00-4a1a-82a9-6b6322b93b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:02.894003 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-07fec1cf-bb00-4a1a-82a9-6b6322b93b34 None None] MFA Rules not processed for user `4d0f702c6dd04ac1a8cf16b1ca420f19`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:56:02.895199 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-07fec1cf-bb00-4a1a-82a9-6b6322b93b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:02.914939 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:02.954420 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1458/2901] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:56:02.964067 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:02.966474 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:03.022436 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dda90-b815-4bf5-ad4d-d0fe197f60a5 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:03.026032 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c54c5182ce245e49cdbf25f1254238e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:03.026641 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:03.026928 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:03.027922 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:03.036845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:03.038322 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:03.047846 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1444/2902] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:56:04.201122 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776a18-e479-4e6e-b2e0-7ccdb4a5689e None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:04.201750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776a18-e479-4e6e-b2e0-7ccdb4a5689e None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:04.201750 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd776a18-e479-4e6e-b2e0-7ccdb4a5689e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:04.232374 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-fd776a18-e479-4e6e-b2e0-7ccdb4a5689e None None] MFA Rules not processed for user `47c57945a0bb4fc78a812b7d453fca81`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:56:04.233730 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fd776a18-e479-4e6e-b2e0-7ccdb4a5689e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:04.298963 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1459/2903] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:56:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:56:04.309560 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:04.310980 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:04.368123 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb3cbc-13d6-4824-9c41-f7447841f37e service neutron] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:04.371991 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '428e53d5cb9e4a2996295b94e364ece4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:04.372596 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:04.372873 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:04.373149 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:04.381367 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:04.382794 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:04.392293 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1445/2904] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:56:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:56:06.195306 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:06.195819 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:06.195935 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:06.215517 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] MFA Rules not processed for user `47c57945a0bb4fc78a812b7d453fca81`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:56:06.216722 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:06.223645 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1460/2905] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:56:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:56:06.826989 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:06.828519 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:06.900374 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9170559-deea-49c9-a700-5d8cbc43e00a service cinder] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:06.906084 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:06.906940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:06.906940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:06.907224 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:06.908965 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:06.920679 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:06.922290 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:06.932157 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1446/2906] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:56:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:56:16.209580 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=50 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:16.514822 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:56:16.516255 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:16.590225 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb99a8-16f3-40a5-a0e6-3ea076923982 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:56:16.595917 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:56:16.596542 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:56:16.596832 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:56:16.597147 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:56:16.598911 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:56:16.612004 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:56:16.613640 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98273b8-d302-457e-8112-73f1f51c1362 service nova] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:56:16.624699 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1461/2907] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:56:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:56:16.927940 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:56:26.619847 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=34 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:57:14.158766 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:57:14.174711 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7420ef-6754-4885-8b59-8a9250faf2e5 service placement] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:57:14.180175 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09fbb54e522c47d89de166682742336f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:57:14.180808 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:57:14.181090 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:57:14.181399 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:57:14.183200 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:57:14.196721 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:57:14.198349 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:57:14.210247 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1447/2908] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:57:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:57:24.205063 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=6 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:57:25.823886 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] REQUEST_METHOD: `POST` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:57:25.824399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:57:25.824399 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:57:25.854462 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.auth.core [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] MFA Rules not processed for user `47c57945a0bb4fc78a812b7d453fca81`. Rule list: `[]` (Enabled: `True`). {{(pid=80713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 03:57:25.855646 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:57:25.879781 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1462/2909] 10.222.0.66 () {58 vars in 958 bytes} [Thu Aug 4 03:57:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 03:57:35.877768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=15 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:58:34.238597 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:58:34.300145 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b8f7-72ba-46bb-bb22-1894d80d261e service nova] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:58:34.305274 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:58:34.305883 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:58:34.306171 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:58:34.306446 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:58:34.308334 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:58:34.332891 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:58:34.334607 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:58:34.344700 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1448/2910] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:58:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:58:34.382559 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:58:34.410126 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53048675-96c5-41a0-99ec-790a1e77c4f0 None None] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:58:34.414744 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:58:34.415381 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:58:34.415668 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:58:34.415937 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] PATH_INFO: `/v3/users/47c57945a0bb4fc78a812b7d453fca81` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:58:34.417113 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c57945a0bb4fc78a812b7d453fca81)` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:58:34.419084 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:58:34.580160 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c57945a0bb4fc78a812b7d453fca81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:34.587674 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c57945a0bb4fc78a812b7d453fca81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:34.588336 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c57945a0bb4fc78a812b7d453fca81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:34.600195 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c57945a0bb4fc78a812b7d453fca81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:34.693804 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.notifications [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c57945a0bb4fc78a812b7d453fca81'} {{(pid=80713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:34.699869 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1463/2911] 10.222.0.66 () {58 vars in 1224 bytes} [Thu Aug 4 03:58:34 2022] DELETE /identity/v3/users/47c57945a0bb4fc78a812b7d453fca81 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 03:58:34.930368 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:58:34.942948 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7e8e2-00a8-4fe1-bee5-0da678d24e13 service placement] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:58:35.002435 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d0f702c6dd04ac1a8cf16b1ca420f19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '94f13981caf746e383601ed3304a94e7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:58:35.003126 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:58:35.003408 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:58:35.003683 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] PATH_INFO: `/v3/projects/16033dcf8c904cb2af2c8701a4b6a716` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:58:35.004848 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16033dcf8c904cb2af2c8701a4b6a716)` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:58:35.006800 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:58:35.123615 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16033dcf8c904cb2af2c8701a4b6a716', 'request_id': 'req-c6ba8091-a661-4ce5-a34f-c0d90180f663'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:35.129925 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16033dcf8c904cb2af2c8701a4b6a716', 'request_id': 'req-c6ba8091-a661-4ce5-a34f-c0d90180f663'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:35.196752 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16033dcf8c904cb2af2c8701a4b6a716', 'request_id': 'req-c6ba8091-a661-4ce5-a34f-c0d90180f663'} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:35.219999 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] The token cache is being invalidate because project 16033dcf8c904cb2af2c8701a4b6a716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80714) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 03:58:35.220287 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:35.220917 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.notifications [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80714) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 03:58:35.223221 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1449/2912] 10.222.0.66 () {58 vars in 1233 bytes} [Thu Aug 4 03:58:34 2022] DELETE /identity/v3/projects/16033dcf8c904cb2af2c8701a4b6a716 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 03:58:40.281495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Authenticating user token {{(pid=80713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 03:58:40.281495 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:58:40.325896 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54f904-ee8e-4847-857a-796671f207f9 admin admin] Validating token access rules against request {{(pid=80713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 03:58:40.377768 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:58:40.385721 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 03:58:40.390788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] REQUEST_METHOD: `GET` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 03:58:40.390788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 03:58:40.390788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 03:58:40.390788 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.fernet_utils [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 03:58:40.400532 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 03:58:40.402177 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cf3f1-81d5-46bf-81a8-dab1e30b08c9 service placement] RBAC: Authorization granted {{(pid=80713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 03:58:40.412290 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: [pid: 80713|app: 0|req: 1464/2913] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 03:58:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 03:58:45.217710 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 03:58:50.469666 dv-cinder-733062-32-iscsi devstack@keystone.service[80713]: DEBUG dbcounter [-] [80713] Writing DB stats keystone:SELECT=5 {{(pid=80713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 04:03:45.261764 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Authenticating user token {{(pid=80714) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 04:03:45.263163 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.fernet_utils [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80714) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 04:03:45.278127 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba8091-a661-4ce5-a34f-c0d90180f663 admin admin] Validating token access rules against request {{(pid=80714) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 04:03:45.283006 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35673f56-0b72-4152-a3a2-d2cb9a8a481d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce4b77895c73479c9811fa2fd6d7ed71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cdc5a6c10712453980a6cf3426011060', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80714) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 04:03:45.283606 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673f56-0b72-4152-a3a2-d2cb9a8a481d service placement] REQUEST_METHOD: `GET` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 04:03:45.283889 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673f56-0b72-4152-a3a2-d2cb9a8a481d service placement] SCRIPT_NAME: `/identity` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 04:03:45.284166 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35673f56-0b72-4152-a3a2-d2cb9a8a481d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80714) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 04:03:45.293251 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35673f56-0b72-4152-a3a2-d2cb9a8a481d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 04:03:45.294845 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35673f56-0b72-4152-a3a2-d2cb9a8a481d service placement] RBAC: Authorization granted {{(pid=80714) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 04:03:45.305463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: [pid: 80714|app: 0|req: 1450/2914] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Aug 4 04:03:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 04:03:55.300463 dv-cinder-733062-32-iscsi devstack@keystone.service[80714]: DEBUG dbcounter [-] [80714] Writing DB stats keystone:SELECT=6 {{(pid=80714) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}